HAUFE CONTENT HUB
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 2
› Central content discovery and exchange platform
› within the Haufe Group and
› with partners.
› Developed under new Haufe Technology Strategy
› First release planned summer 2016
SCENE SETTING: HAUFE GROUP
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 3
Akademie Trainings,
e-Learnings
CCP Consultants,
Corporations & Public Sector
SBC Small
Businesses & Consumer
RLN Real Estate, Lawyers & Non Profit
H2/H3 Product
Innovation
Umantis HR Software
LSL Literature
Procurement
VCW Publisher
(Controlling)
HLRE Real Estate Software
Hammonia Publisher
(Real Estate)
Haufe Inc. US Market (HR)
New Times Corporate Publishing
Schäffer-Poeschel
Publisher (Taxes, Law, Economics)
Smart-steuer
Taxes SaaS
Semigator Trainings
Procurement
HAUFE: THE ROOTS
Christoph Ludwig: Haufe Content Hub – Turn Content into APIs
Books
Loose-leaf editions
Desktop content databases (1990s)
2016-06-14 Seite 4
HAUFE TODAY
Christoph Ludwig: Haufe Content Hub – Turn Content into APIs
Online Content Databases Haufe.de News Site
Booking platforms for seminars & trainings
Books & Print Products
2016-06-14 Seite 5
SaaS solutions (accounting, tax, HR, …)
… and more!
Academy Shop Content
SBC Online Trainings
CCP Online Trainings
SAME OLD STORY: CONTENT SILOS EVERYWHERE
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 6
Bulk-produced Content Products
Subject-specific News
Partner Content
Academy Training Materials
Customer Content
SBC Shop Content
CCP Shop Content
Academy Shop Content
SBC Online Trainings
CCP Online Trainings
WELL-TRIED APPROACH: CONTENT HUB
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 7
Bulk-produced Content Products
Subject-specific News
Partner Content
Academy Training Materials
Customer Content
SBC Shop Content
CCP Shop Content
Content Hub
Composable Enterprise
DIGRESSION: HAUFE TECH STRATEGY
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 8
Evolvable
Rapid Onboarding
Operation at Scale
Efficiency at Scale
Open to Partners
Customer-centric
Interoperability
STRATEGY IMPLEMENTATION: APIS
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 9
Ease Integration
Partnership Development, Extend Reach
Innovate through Product Unbundling
Decouple Service Consumers and Providers
New Business Models
Insight and Control over API success and usage
APIs
CONTENT HUB BUILDING BLOCKS
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 10
Content Producer Content Consumer
API Gateway
Content Hub Implementation
Ingest API Search / Retrieval API
Cross-cutting concerns: › Authorization › Rate limiting & metering › Usage analytics › …
CONTENT HUB BUILDING BLOCKS
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 11
Bulk Content Producer Content Consumer
API Gateway
Content Hub Implementation
Search / Retrieval API
Incremental Content Producer
Bulk Ingest API
Single Doc Ingest API
File Storage API
CONTENT HUB BUILDING BLOCKS
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 12
Bulk Content Producer Content Consumer
API Gateway Bulk Ingest API
Search / Retrieval API
Single Doc Ingest API
Incremental Content Producer
Ingest Service Search & Retrieval Service
Admin UI Tmp Bulk
File Storage
File Storage API
CONTENT HUB API PRINCIPLES
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 13
Guiding Principles:
› Outside-in approach: Client convenience drives the API, don’t leak implementation details!
› Ease of use: Make client developers quickly productive:
› adhere to common conventions;
› provide reasonable defaults;
› provide developer guides and comprehensive documentation;
› Self-descriptive: Clients learn about potential next steps from embedded hypermedia controls.
Public at http://dev.haufe.com/api-style-guide/
CLIENT-DRIVEN SEARCH API
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 14
› “Customer” interviews:
› Standard features (full-text search, wildcards, constraints, faceting, pagination, …)
› Group-by / result aggregation
› Suggest
› Search expression language:
› Inspired by well-known search APIs
› Lends itself to OpenSearch descriptions
› Results returned as OpenSearch v1.1 response based on Atom collections
Hit # Doc Id Score Property
1 23.4.67 0.9746 a
2 20.1.08 0.9255 b
3 43.9.64 0.8843 a
4 30.9.53 0.8802 c
5 20.2.30 0.7945 b
6 18.7.42 0.7769 d
Hit # Doc Id Score Property
1 (1) (3)
23.4.67 43.9.64
0.9746 0.8843
a a
2 (2) (5) (9)
20.1.08 20.2.30 21.5.23
0.9255 0.7945 0.5810
b b b
3 (4) 30.9.53 0.8802 c
CONTENT HUB SINGLE DOC INGEST API
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 15
› Synchronous
› Based on the Atom Publishing Protocol
› Every producing application has its own set of Atom Pub feeds.
› Document ingest uses the usual Atom publishing operations.
› UBER (Uniform Basis for Exchanging Representations) data embedded into Atom service document provides request template for Atom extension (document lookup).
› Supports (but does not require) XML-binary Optimized Packaging (XOP) for attached blobs.
Single Doc Ingest Service Document
Producer App A Atom Workspace
Producer App B Atom Workspace
App A Content Docs
Atom Collection
App B Content Docs
Atom Collection
App A Contract Docs
Atom Collection
App B Contract Docs
Atom Collection
BULK INGEST API
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 16
› Asynchronous
› Bulk ZIP archives: Multiple GByte (up to millions of XML documents + some binaries)
› Producer does not necessarily control publication
Bulk Content Producer
Ingest Service
Tmp Bulk File Storage
API
Gat
eway
Bul
k In
gest
API
Fi
le S
tora
ge
API
Publisher
1. Create ingest job Response: job ID, storage URL
2. Upload archive to storage URL
3. Approve ingestion
4. Ingest archive when scheduled
DEPLOYMENT & OPERATIONS
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 17
Composable Enterprise
Evolvable
Rapid Onboarding
Operation at Scale
Efficiency at Scale
Open to Partners
Customer-centric
Interoperability
APIs
APIs APIs
APIs APIs
Tech Strategy: Efficiency and operation at scale by
› fully automated
› cloud deployments.
In May 2016, Haufe Api Management Deployment took
15 secs
WHAT DO WE WANT TO ACHIEVE?
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 18
Speed
Business Value In Manual Deployments?
Reduce Error
Prepare For Change
Make it repeatable Write it down Include testing Start early in the process Automate
Hoster, on premise, cloud(s) Different technologies Business scenarios change
AZURE DEPLOYMENT: Building Blocks Become…
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 19
API Gateway Bulk Ingest API
Search / Retrieval API
Single Doc Ingest API
Ingest Service Search & Retrieval Service
Admin UI Tmp Bulk
File Storage
File Storage API
Doc
ker
Hos
t Ingest Service
Search & Retrieval Service
Admin UI
Doc
ker
Hos
t Ingest Service
Search & Retrieval Service
Admin UI
Doc
ker
Hos
t Ingest Service
Search & Retrieval Service
Admin UI
AZURE CLOUD DEPLOYMENT
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 20
Azure API Mgmt Gateway Bulk Ingest API
Search / Retrieval API
Single Doc Ingest API
Azure Blob Storage Azure Premium Storage
MarkLogic VM
Azure Premium Storage
MarkLogic VM
Azure Premium Storage
MarkLogic VM Stateless Docker
Containers w/ Spring Boot Apps
IaaS Offerings
PaaS Offerings
DEPLOYMENT AUTOMATION W/ GO.CD
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 21
Maven Repo Docker Registry
Build Service Jars
Build Docker Images
Deploy Test Env
Deploy Prod Env
Build Base VM Images
Azu
re
Azure VM Images
Content Hub Test Environment
Content Hub Prod Environment
Security Scan
SUMMARY
2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 22
Exposing APIs only: › Decoupled Content Hub implementation from clients;
› Freed implementation from cross-cutting requirements (authorization, rate limiting, usage analytics etc.)
› Simplifies on-boarding of new clients and reaching out to new partners
Fully automated cloud deployments: › Developed in collaboration with operations partner Ebcont;
› Up-front effort, expected to amortize soon;
› Reproducible environments;
› Unburdens dev / ops team;
› Security scans shifted into the build pipeline;
› Precondition for operating and evolve the application at scale (with a small team)