Hello elixir (and otp)

HELLO ELIXIR (AND OTP)Abel Muiño (@amuino) & Rok Biderman (@RokBiderman)

http://twitter.com/amuino

http://twitter.com/RokBiderman

ULTRA SHORT INTROS

Abel Muiño Lead developer at Cabify. Works with ruby for a living.

[email protected] / @amuino

Rok Biderman Senior Go developer at Cabify. Has an interesting past position. Go ask him.

[email protected] / @RokBiderman

mailto:[email protected]

mailto:[email protected]

WE ARE HIRINGRuby, Go, Javascript, Android, iOS

(Just not for Elixir, yet)

HELLO ELIXIR (AND OTP)Abel Muiño (@amuino) & Rok Biderman (@RokBiderman)



GOALS

➤ Show some code, this is a programming meet up

➤ Share our Elixir learning path

➤ Learn something from feedback and criticism

➤ Hopefully at least one other person will learn one thing

“This is not production code

-Abel Muiño

BUILDING AN OCR MODULE

Extracting quotes from memes

TL;DR http://github.com/amuino/ocr

http://github.com/amuino/ocr

MIX NEW

$ mix new ocr* creating README.md* creating .gitignore* creating mix.exs* creating config* creating config/config.exs* creating lib* creating lib/ocr.ex* creating test* creating test/test_helper.exs* creating test/ocr_test.exs

Your Mix project was created successfully.You can use "mix" to compile it, test it, and more:

cd ocr mix test

Run "mix help" for more commands.

MIX NEW



cd ocr mix test


Project Definition

MIX NEW



cd ocr mix test


Project Definition

App config

MIX NEW



cd ocr mix test


Project Definition

App config

Main module

MIX NEW



cd ocr mix test


Project Definition

App config

Main module

😓Not talking about tests today

THE INTERACTIVE SHELL

$ iex -S mixErlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Compiled lib/ocr.exGenerated ocr appConsolidated List.CharsConsolidated CollectableConsolidated String.CharsConsolidated EnumerableConsolidated IEx.InfoConsolidated InspectInteractive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)> OcrOcr




Automatically compiles new files





Lots of first-run noise





Lots of first-run noise

SUCCESS! Our module exists

💡 TIP: TRAINING WHEELS

➤ We are learning, so getting quick feedback is useful

➤ Credo is a static code analysis tool for the Elixir language with a focus on teaching and code consistency. https://github.com/rrrene/credo

➤ Credo installs as a project dependency

➤ Adds a new task to mix to analyse our code

➤ Excellent, very detailed, feedback

https://github.com/rrrene/credo

ADD A DEPENDENCY

Find the dependency info in hex.pmEdit mix.exs defp deps do [ {:credo, "~> 0.3", only: [:dev]} ] end

Install it locally$ mix geps.get

ADD A DEPENDENCY



Dependencies are an array of tuples.

ADD A DEPENDENCY



Dependencies are an array of tuples.

Only installs the dependency in the :dev

environment

TRY IT

$ mix credo… Code Readability [R] ! Modules should have a @moduledoc tag. lib/ocr.ex:1:11 (Ocr)

$ mix credo lib/ocr.ex:1:11

TRY IT



OMG!

TRY IT



OMG!

Detailed explanation on the error, how to suppress it,

etc…

NOW WHAT?

➤ Use Google Vision API to perform the actual OCR

➤ Has no client in hex.pm

➤ It is a REST API → {:httpoison, "~> 0.8.3"}

➤ Returns JSON → {:poison, "~> 2.1.0"}

➤ Needs authentication → {:goth, "~> 0.1.2”}

➤ Build a nice façade

MIX.EXS

def application do [applications: [:logger, :httpoison, :goth]] end

defp deps do [ {:httpoison, "~> 0.8.3"}, {:poison, "~> 2.1.0"}, {:goth, "~> 0.1.2"}, {:credo, "~> 0.3", only: [:dev]} ] end

MIX.EXS

def application do [applications: [:logger, :httpoison, :goth]] end

defp deps do [ {:httpoison, "~> 0.8.3"}, {:poison, "~> 2.1.0"}, {:goth, "~> 0.1.2"}, {:credo, "~> 0.3", only: [:dev]} ] end

Some deps also need their app to be started

CONFIG/CONFIG.EXS

use Mix.Config

config :goth, json: "config/google-creds.json" |> File.read!

CONFIG/CONFIG.EXS

use Mix.Config

config :goth, json: "config/google-creds.json" |> File.read!

Some deps also have their own config

NOW WHAT?

➤ We will write 2 modules:

➤ Ocr.GoogleVision for the API client.

➤ Ocr for our façade

💡 TIP: MODULE NAMES

➤ Convention: ➤ Ocr ! lib/ocr.ex➤ Ocr.GoogleVision ! lib/ocr/google_vision.ex

➤ Modules names are just names. Dots in the name do not represent any parent/child relationship.

LIB/OCR/GOOGLE_VISION.EX

defmodule Ocr.GoogleVision do def extract_text(image64) do image64 |> make_request |> read_body end # MAGIC!end



base64 encoded image




send to Google




send to Google

get the text from the response


@url "https://vision.googleapis.com/v1/images:annotate"@feature_text_detection "TEXT_DETECTION"@auth_scope "https://www.googleapis.com/auth/cloud-platform"

def make_request(image64) do HTTPoison.post!(@url, payload(image64), headers)end

defp payload(image64) do %{requests: [ %{image: %{content: image64}, features: [%{type: @feature_text_detection}]} ] } |> Poison.encode!end

defp headers do {:ok, token} = Goth.Token.for_scope(@auth_scope) [{"Authorization", "#{token.type} #{token.token}"}]end






Module attributes (used as a constants)







HTTP POST some JSON to some URL with some Headers








The JSON Google wants









Get a token









Get a token

Put the token on the request headers


def read_body(%HTTPoison.Response{body: body, status_code: 200}) do body |> Poison.decode! |> get_in(["responses", &first/3, "textAnnotations", &first/3, "description"])end

defp first(:get, nil, _), do: nildefp first(:get, data, next) do data |> List.first |> next.()end




Only care about body and success http status





Parse JSON response





Parse JSON response

Extract the text





Parse JSON response

Extract the text

Custom lookup functions





Parse JSON response

Extract the text

Custom lookup functions

funky syntax to invoke an anonymous function

💡 TIP: GET_IN IS AWESOME

➤ Navigates nested structures (maps)

iex> users = %{"john" => %{age: 27}, "meg" => %{age: 23}}iex> get_in(users, ["john", :age])27➤ Returns nil on missing keys

iex> get_in(users, ["unknown", :age])nil➤ Accepts functions for navigating other types

➤ Elixir 1.3 will have common functions predefined for tuples and lists

➤ Access.at(0) will replace my custom first (PR #4719)

➤ Also check get_and_update_in, put_in, update_in

https://github.com/elixir-lang/elixir/pull/4719

LIB/OCR.EX

defmodule Ocr do def from_base64(b64), do: Ocr.GoogleVision.extract_text(b64)

def from_image(image_data) do image_data |> Base.encode64 |> from_base64 end

def from_path(path), do: path |> File.read! |> from_image

def from_url(url), do: HTTPoison.get!(url).body |> from_imageend

LIB/OCR.EX





LIB/OCR.EX





LIB/OCR.EX





FUN!


A new Hex version is available (0.12.0), please update with `mix local.hex`Interactive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)> meme_url = "http://ih0.redbubble.net/image.16611809.2383/fc,550x550,black.jpg""http://ih0.redbubble.net/image.16611809.2383/fc,550x550,black.jpg"iex(2)> IO.puts Ocr.from_url meme_urlGETS ELIKIR PR ACCEPTEDI SAID WHO WANTS TOFUCKING TOUCH ME?Suranyami

:ok

FUN!


A new Hex version is available (0.12.0), please update with `mix local.hex`Interactive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)> meme_url = "http://ih0.redbubble.net/image.16611809.2383/fc,550x550,black.jpg""http://ih0.redbubble.net/image.16611809.2383/fc,550x550,black.jpg"iex(2)> IO.puts Ocr.from_url meme_urlGETS ELIKIR PR ACCEPTEDI SAID WHO WANTS TOFUCKING TOUCH ME?Suranyami

:ok

STATEFUL? STATELESS?Cast your vote!

ANSWER: STATEFUL

➤ Auth tokens are not requested every time

➤ Requested on first use

➤ Refreshed on the background when about to expire

➤ Goth.TokenStore is a GenServer➤ Just one of the predefined behaviours to make easier to

work with processes

➤ Starts a process with some state

➤ Receives messages and updates the state

➤ There is more state (like Goth.Config)

DEPS/GOTH/LIB/GOTH/TOKEN_STORE.EX

defmodule Goth.TokenStore do use Xenserver alias Goth.Token

def start_link do GenServer.start_link(__MODULE__, %{}, [name: __MODULE__]) end

def handle_call({:store, scope, token}, _from, state) do pid_or_timer = Token.queue_for_refresh(token) {:reply, pid_or_timer, Map.put(state, scope, token)} end

def handle_call({:find, scope}, _from, state) do {:reply, Map.fetch(state, scope), state} endend






Start the process,






Start the process, Initial state is an empty map







The process has a name







The process has a name

Handle 2 types of messages, returning something

FUN WITH TOKEN STORE

$ iex -S mixiex(1)> token = %Goth.Token{token: "FAKE", expires: :os.system_time + 10_000_000}%Goth.Token{expires: 1464647952951907000, scope: nil, token: "FAKE", type: nil}

iex(2)> GenServer.call Goth.TokenStore, {:find, "Elixir"} :error

iex(3)> GenServer.call Goth.TokenStore, {:store, "Elixir", token}{:ok, {1464647950910798703208727, #Reference<0.0.7.228>}}

iex(4)> GenServer.call Goth.TokenStore, {:find, "Elixir"}{:ok, %Goth.Token{expires: 1464647952951907000, scope: nil, token: “FAKE", type: nil}}






The process name






The process name

The message


defmodule Goth.TokenStore do

def store(%Token{}=token), do: store(token.scope, token) def store(scopes, %Token{} = token) do GenServer.call(__MODULE__, {:store, scopes, token}) end

def find(scope) do GenServer.call(__MODULE__, {:find, scope}) endend


defmodule Goth.TokenStore do

def store(%Token{}=token), do: store(token.scope, token) def store(scopes, %Token{} = token) do GenServer.call(__MODULE__, {:store, scopes, token}) end

def find(scope) do GenServer.call(__MODULE__, {:find, scope}) endend

Provide a client API (usually in the same module) with nicer methods hiding the use of

Genserver.call

LET’S TALK ABOUT OTPProcesses, state, concurrency, supervisors,… oh my!

RECAP

➤ Erlang is:

➤ general-purpose

➤ concurrent

➤ garbage-collected

➤ programming language

➤ and runtime system

➤ Elixir:

➤ Builds on top of all that

START A PROCESS

➤ Basic concurrency primitive

➤ Simplest way to create, use spawn with a function defmodule BasicMessagePassing.Call do def concat(a, b) do IO.puts("#{a} #{b}") endend

iex(2)> BasicMessagePassing.Call.concat "Elixir", "Madrid"Elixir Madrid:okiex(3)> spawn BasicMessagePassing.Call, :concat, ["Elixir", "Madrid"]Elixir Madrid#PID<0.69.0>

START A PROCESS




Same process

START A PROCESS




Same process

Spawned process id

LISTEN FOR MESSAGES

defmodule BasicMessagePassing.Listen do def listen do receive do {:ok, input} -> IO.puts "#{input} Madrid" end end end

iex(5)> pid = spawn(BasicMessagePassing.Listen, :listen, [])#PID<0.82.0>iex(6)> send pid, {:ok, "Elixir"}Elixir Madrid{:ok, "Elixir"}iex(8)> Process.alive? pidfalse

FIBONACCI TIME!defmodule FibSerial do def calculate(ns) do ns |> Enum.map(&(calc(&1))) |> inspect |> IO.puts end

def calc(n) do calc(n, 1, 0) end

defp calc(0, _, _) do 0 end

defp calc(1, a, b) do a + b end

defp calc(n, a, b) do calc(n - 1, b, a + b) end end FibSerial.calculate(Enum.to_list(1..10000))

FIBONACCI TIME!defmodule FibSerial do def calculate(ns) do ns |> Enum.map(&(calc(&1))) |> inspect |> IO.puts end

def calc(n) do calc(n, 1, 0) end

defp calc(0, _, _) do 0 end

defp calc(1, a, b) do a + b end

defp calc(n, a, b) do calc(n - 1, b, a + b) end end FibSerial.calculate(Enum.to_list(1..10000)) About 6 seconds

PARALLEL FIBONACCI TIME!defmodule FibParallel do def calculate(ns) do ns |> Enum.with_index |> Enum.map(fn(ni) -> spawn FibParallel, :send_calc, [self, ni] end) listen(length(ns), []) end

def send_calc(pid, {n, i}) do send pid, {calc(n), i} end

defp listen(lns, result) do receive do fib -> result = [fib | result] if lns == 1 do result |> Enum.sort(fn({_, a}, {_, b}) -> a < b end) |> Enum.map(fn({f, _}) -> f end) |> inspect |> IO.puts else listen(lns - 1, result) end end end end FibSerial.calculate(Enum.to_list(1..10000))

PARALLEL FIBONACCI TIME!defmodule FibParallel do def calculate(ns) do ns |> Enum.with_index |> Enum.map(fn(ni) -> spawn FibParallel, :send_calc, [self, ni] end) listen(length(ns), []) end

def send_calc(pid, {n, i}) do send pid, {calc(n), i} end

defp listen(lns, result) do receive do fib -> result = [fib | result] if lns == 1 do result |> Enum.sort(fn({_, a}, {_, b}) -> a < b end) |> Enum.map(fn({f, _}) -> f end) |> inspect |> IO.puts else listen(lns - 1, result) end end end end FibSerial.calculate(Enum.to_list(1..10000)) About 2 seconds (4 cores)

LINKING PROCESSES

➤ If child dies, parent dies defmodule BasicMessagePassing.Linking do def exit, do: exit(:crash)

def start do spawn_link(BasicMessagePassing.Linking, :exit, []) receive do {:done} -> IO.puts "no more waiting" end end end

iex(13)> BasicMessagePassing.Linking.start** (EXIT from #PID<0.57.0>) :crash

Interactive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)>

LINKING PROCESSES

➤ If child dies, parent dies defmodule BasicMessagePassing.Linking do def exit, do: exit(:crash)

def start do spawn_link(BasicMessagePassing.Linking, :exit, []) receive do {:done} -> IO.puts "no more waiting" end end end

iex(13)> BasicMessagePassing.Linking.start** (EXIT from #PID<0.57.0>) :crash

Interactive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)>

iex died! and was restarted

LINKING PROCESSES

➤ If child dies, parent dies… unless it handles the dead defmodule BasicMessagePassing.Linking do def exit, do: exit(:crash)

def start do Process.flag(:trap_exit, true) spawn_link(BasicMessagePassing.Linking, :exit, []) receive do {:EXIT, from_pid, reason} -> IO.puts "#{inspect(self)} is aware #{inspect(from_pid)} exited because of #{reason}" end end end

iex(24)> BasicMessagePassing.Linking.start#PID<0.57.0> is aware #PID<0.157.0> exited because of crash:ok

LINKING PROCESSES




survive children

LINKING PROCESSES




survive children

handle deads

GENSERVER

➤ Simplifies all this stuff

➤ Is a process like any other Elixir process

➤ Standard set of interface functions, tracing and error reporting

➤ call: request with response

➤ cast: request without response

A STACK

defmodule Stack do use GenServer

def start_link(state, opts \\ []) do GenServer.start_link(__MODULE__, state, opts) end

def handle_call(:pop, _from, [h|t]) do {:reply, h, t} end

def handle_cast({:push, h}, t) do {:noreply, [h|t]} endend

A SUPERVISED STACKiex(7)> import Supervisor.Specniliex(8)> children = [...(8)> worker(Stack, [[:first], [name: :stack_name]])...(8)> ][{Stack, {Stack, :start_link, [[:first], [name: :stack_name]]}, :permanent, 5000, :worker, [Stack]}]iex(9)> {:ok, pid} = Supervisor.start_link(children, strategy: :one_for_one){:ok, #PID<0.247.0>}iex(10)> GenServer.call(:stack_name, :pop):firstiex(11)> GenServer.call(:stack_name, :pop)18:04:35.012 [error] GenServer :stack_name terminating** (FunctionClauseError) no function clause matching in Stack.handle_call/3 iex:13: Stack.handle_call(:pop, {#PID<0.135.0>, #Reference<0.0.1.317>}, [])[..]Last message: :popState: [] (elixir) lib/gen_server.ex:564: GenServer.call/3iex(11)> GenServer.call(:stack_name, :pop):first

SUPERVISOR FLAVORS

➤ one_for_one: dead worker is replaced by another one

➤ rest_for_all: after one dies, all others have to be restarted

➤ rest_for_one: all workers started after this one will be restarted

➤ simple_one_for_one: for dynamically attached children, Supervisor is required to contain only one child

QUESTIONS?

THANKS!Abel Muiño (@amuino) & Rok Biderman (@RokBiderman)



Date post:	22-Jan-2018
Category:	Software
Upload:	abel-muino
View:	588 times
Download:	0 times

Hello elixir (and otp)

Software