Clojure Finland 10.3.2016Tommi Reiman / Metosin

@ikitommi

Schema-tools & tricks

Schema-tools & tricks& quick clojure.spec intro

Clojure Finland 10.3.2016 22.6.2016Tommi Reiman / Metosin

@ikitommi

Schema• https://github.com/plumatic/schema• Structural data validation library for Clojure(Script)

– Based on predicates, (almost) just data• Runtime type-checking• Documentation for data & functions• Recursive transformations with coercion and matchers• Macros for defining schematized functions• Data generation & completion

• DEMO: basic syntax

Schema is real life projects (~all of ours)

• (Optionally) annotate your domain functions and values with Schema– Schema def , defschema & defn– Plumbing defnk , fnk & letk

• Enable schema checking for tests (catching 90% of type bugs)

• Turn off optional validation in production• Force coercion when needed / at the system boundaries• Data coercion between contexts

Schemas in web development• Schema coercions when reading from external sources (JSON, XML,

db, …)• Schema-based remote apis

– Ring-swagger (https://github.com/metosin/ring-swagger)– Used by compojure-api, fnhouse-swagger, kekkonen, pedestal-api, yada

• Schemas for the ClojureScript frontend– Code sharing via cljc (common domain: both data & function to operate on ’em)– schema-tools for combining ui-views & server-views

• End2end schemas for Clojure(Script)?– Dynamic Schemas over the wire with Transit (allows only partial validation)– Resolve the expression problem (new wire-formats, new types)

Schema example (with kekkonen.io)

Schema-tools (https://github.com/metosin/schema-tools)

• Selectors, transformers, walkers• Handling of defaults• Coercion helpers

• DEMO: cut & paste models• DEMO: declarative transformations• DEMO: defaults• DEMO: coercion speed (case RegistryX)

Schema-viz (https://github.com/metosin/schema-viz)

Schema takeaways• Schema is awesome for runtime structural validation – for both

system boundaries & tests• Noise-free syntax with schematized data & function definitions• Has been de facto in Clojure web (api) development• Mostly just data, enabling cool things like declarative

transformations

clojure.spec(a quick overview)

What is• a new Integrated system for specification and testing, shipped with Clojure 1.9• Automatic validation, error reporting, test-data & test generation• Specs can be written external to the actual functions & data (like core.typed)• Build from ground-up, recursive, regexps• (Mutable) Global spec registry• Form, Conform & Unform, multi-spec• Function specs on :args, :ret and :fn(!)• Opinions

– Maps should be open– Namespaced keywords– Verification via (generative) tests

• Currently is not– A runtime validation system– In alpha6, the :ret and :fn was removed from standard instrumentation, hmph.

CODE

Function specs

https://clojure.org/guides/spec

+spec forhigher order functions+spec formacros+multispec formultimethods

Spec for runtime validation?

https://clojure.org/guides/spec

Coercion?

• Conformers live within the spec (vs external with Schema)

• How can I do custom conform based on external info (e.g. protocol used)

Generating data

https://clojure.org/guides/spec

Generative testing

https://clojure.org/guides/spec

More info about spec• https://clojure.org/guides/spec• Slack #clojure-spec (both Rich & Alex seem to hang

out there)• Changes in the language

– Lot’s of new predicates in clojure.core– Namespaced maps http://dev.clojure.org/jira/browse/CLJ-1910– Destructuring http://dev.clojure.org/jira/browse/CLJ-1919

Schema, Spec vs Both?

Performance2-6xfaster

Notoptimizedforperf,yet.http://muhuk.github.io/validation-benchmark/

Differences• Spec has better support for language level

features like multi-methods and higher order functions

• Schema supports closed maps & coercion• Both support data generation & generative

tests• Schema is not found by Rich Hickey

Specs for the web, today?• Played with a idea to combine the two:

– Convert a pragmatic subset of spec predicates into Schema Predicates • integer? => s/Int• boolean? => s/Bool• (and integer? pos?) => (s/constrained s/Int pos?)

– Use Schema as before (web-libs, snappy coercion etc.)è https://github.com/metosin/spec-tools & integrate into current apis

My 2 cents• Schema is currently much better for runtime validation (at borders)

– Support (api-docs, api-libs), coercions, performance, low ceremony

• Spec will become de facto way for specifying Clojure systems as it will be part of the core– Under active development, let’s enjoy the ride!– Better runtime validation support via 3rd party libs

• Low ceremony function & data definitions via extra macros• Coercions(!)

– Tooling support (IDEs, api-libs)

• At least for now, the can co-exist…èTODAY: Schema for the boundaries, more spec when things freeze down

Thanks!

Links

• Schema https://github.com/plumatic/schema– Schema-tools https://github.com/metosin/schema-tools– Schema-viz https://github.com/metosin/schema-viz– Ring-Swagger https://github.com/metosin/ring-swagger

• Spec http://clojure.org/about/spec– Regular Expressions:

https://en.wikipedia.org/wiki/Regular_expression