Intelligent Device
Management for Distributed
Embedded Systems with
Enea Element
Michael Christofferson
Director Product Marketing, Enea
FOUNDED
1968
TEN OFFICES
IN NORTH
AMERICA,
EUROPE AND
ASIA
REVENUE
67M USD
NO. OF EMPLOYEES
426
Increasing data traffic in communication devices
require new and innovative software solutions to
handle bandwidth, performance and power
requirements.
Enea software is heavily used in wireless
Infrastructure (Macro, small cell), gateway,
terminal, military, auto, etc.
More than 250M of the 325M LTE population
coverage is powered by Enea Solutions
Enea Solutions run in more than 50% of the
world’s 8.2M radio base stations.
Enea has a mature commercial Linux distribution,
built by Yocto, and specially tailored for
networking and communications, with special
emphasis on “REAL-TIME”
Active member of Linaro
Enea has a powerful HA and Device
Management middleware solution on top of Linux
Global presence, global development, and
headquartered in Stockholm, Sweden
Enea - Powering Communications
Numbers for 2011
Connected Systems
Single Node Systems Fixed Multi-Node Systems Private/Public Cloud
Cloud
Typical Node System Architecture
Hardware
App A App B
Framework A
Operating System
Framework B
Framework C
App C
A Typical Compute Node
Linux
Simplifying Assumption:
Linux is the Operating System
Typical Linux System
App
A
App
B
App
C
App
D
Driver
1
Driver
2
Driver
3
File
System
Misc Scripts,
/sys,
/proc,
etc.
Typical Linux System
App
A
App
B
App
C
App
D
Driver
1
Driver
2
Driver
3
File
System
Misc Scripts,
/sys,
/proc,
etc.
Typical Linux System
App
A
App
B
App
C
App
D
Driver
1
Driver
2
Driver
3
File
System
Misc Scripts,
/sys,
/proc,
etc.
Typical Linux System
App
A
App
B
App
C
App
D
Driver
1
Driver
2
Driver
3
File
System
Misc Scripts,
/sys,
/proc,
etc.
Managing a System: Basic Linux
Typical Linux System
App
A
App
B
App
C
App
D
Driver
1
Driver
2
Driver
3
File
System
Misc Scripts,
/sys,
/proc,
etc.
A Single Node System…
…or a Distributed Multi-Node System
Brute Force Management
Edit diverse configuration files
e.g. files in /etc
Touch each service in turn
Restart services and/or entire system
Navigate to each node
Do the above
Coordinate restart of multiple services
Coordinate restart of multiple nodes
Typical Linux System
App
A
App
B
App
C
App
D
Driver
1
Driver
2
Driver
3
File
System
Misc Scripts,
/sys,
/proc,
etc.
Typical Linux System
App
A
App
B
App
C
App
D
Driver
1
Driver
2
Driver
3
File
System
Misc Scripts,
/sys,
/proc,
etc.
Typical Linux System
App
A
App
B
App
C
App
D
Driver
1
Driver
2
Driver
3
File
System
Misc Scripts,
/sys,
/proc,
etc.
Typical Linux System
App
A
App
B
App
C
App
D
Driver
1
Driver
2
Driver
3
File
System
Misc Scripts,
/sys,
/proc,
etc.
Managing a System…At Best, Controlled Chaos
XML-RPC
SNMP
NETCONF
CLI
Web
Serial/SSH
HTML/HTTP
Typical Linux System
App
A
App
B
App
C
App
D
Driver
1
Driver
2
Driver
3
File
System
Misc Scripts,
/sys,
/proc,
etc.
A Single Node System… …or a Distributed Multi-Node System
CLI
Web
CLI
Web
CLI
Web
CLI
Web
Management System
Scripts
Puppet/Chef
Alternate
Purpose-Built Systems Based on Linux
For network devices…or for any connected system
The systems should be more Available
- Particularly if Connected System Provides Network Services
In the world of interoperability and standards, it makes sense to use common management
techniques
Consider a network of
- Network Devices that Provide the Network
- Or Connected Devices that Provide a Solution on top of a Network
There’s a better way to manage these systems than interacting directly with the implementation
System Management: Challenges
CLI
Web EMS/NMS/OSS
text
HTML
SNMP
XML NETCONF
ConsistencyBetween Manager/System
Multiple InterfacesCLI, Web, SNMP, XML, NETCONF
Simple APIFor Applications
Challenges
Reliable Configuration
EMS, NMS, OSS
XML-RPC
SNMP
NETCONF
CLI
Web
Cloud
Embedded System Solutions
Hardware
Linux
Middleware
Applications
Implementation Architecture
Management Model
Embedded Management: Recommendation
B
A
C
D
Operational Data
State
Status
Statistics
Configuration Data
Persistent
Transactions
Ideally it is ACID*
*Atomic, Consistent,
Isolated, Durable
XMLRPC, NETCONF, SNMP
CLI, Web
Notifications
Events
Alarms
Actions
Model Enables ConsistencyBetween Manager/System
EMS, NMS, OSS
Configuration of SystemPersistent ACID Transactions
Viewing Operational Data of SystemState, Status, Statistics
Monitoring System NotificationsTraps, Alarms
Acting on System
Mediation LayerSimplifies Northbound Agent Interface
Embedded Management: Architecture
Modeling
MIB – Management Information Base• Defined using a subset of ASN.1
• A modeling language
• Widely used for SNMP solutions
• Object Indices are scalars
Modeling describes information that is exchanged between communicating systems
UML – Unified Modeling Language• General purpose modeling for software
• Standard graphic notation
• Targeted at object oriented software
• Powerful…complex
CIM – Common Information Model• From DTMF (Dist Mgmt Task Force)
• Apps, networks, databases and devices
• Based on UML YANG• Created for the NETCONF network config protocol
• General enough for other protocols
• Easy for humans to read and write
• Supports definitions of event notifications
• Supports definitions of remote procedure calls
Choose a modeling language and start modeling
XML Schema Definition (XSD)• Alternative to DTD, more useful
• Hard for humans to read and write
• Easy for machines to parse
Traditional Network Management Solution
Database and OO Programming
IT Infrastructure
Modern, Common with Web
Next Generation Network Management
Modeling: Consider YANG
YANG was designed/defined for network management
YANG is simple, easy to read/write with simple tools
YANG supports for modular model definition
YANG, like XML, has natural support for hierarchy
Typically to represent containment rather than inheritance
YANG is extensible
YANG is simple
Embedded Management Service
Object ManagerSession
Manager
Northbound Interface
Datastore
Southbound Interface
Web
CLI
XM
L-R
PC
NE
TC
ON
F
SN
MP
Custo
m
Configuration
• Update
• Validation
Action Requests
• Input
• Output
Operational Data
• State
• Statistics
Notifications
• Events
• Alarms
AAA
Transactions Validation
Client Registration Distribution
Mediation Layer
Ma
nag
ed
Sys
tem
Desired Features
Model Driven
Interactive interfaces (e.g. CLI or Web) are derived and informed by the model
Navigation, multiple choice, context sensitive help
Validation at input – range checking, enumerations, dependencies, consistency
Transactions are validated against the model
Protects configuration of system from being put in an inconsistent or incorrect state
Agreement between Manager and Managed System
Dynamic Configuration
Configuration changes can take effect immediately when applied
Alternative is changing the configuration and restarting the system
Distributed Interface
All software components through the distributed managed system are accessible via the management API
Common Interface
The system software solution is written to a single management service API, regardless of the Northbound interface
Granular Scope of update population and associated update objects
Configuration updates are only distributed to those system clients that are registered for the changed objects
Updates that are provided contain only the subtrees that were registered for
Updates contain indication of what has changed within the registered subtree
Separation of North from South
NETCONF SNMP CLI XML
Application
NETCONF
API
SNMP
API
CLI
API
XML
API
NE
TC
ON
F
SN
MP
CLI
XM
L
NETCONF SNMP CLI XML
Application
EM API
NE
TC
ON
F
SN
MP
CLI
XM
L
EM with Mediation Layer
Stovepipe ArchitectureApplication uses multiple APIs
Unified EM ArchitectureMediation Layer simplifies Application
Addresses Multiple Interfaces and Enabling Simple API
Single Management Interface; Distributed API
The Rest of
Middleware Stack
The Rest of
MW Stack
App AppApp AppApp
AppApp AppApp
The Rest of
MW Stack
AppApp AppApp
The Rest of
MW Stack
AppApp AppApp
OM Data
StoreNotifyAlarm
configuration
status notification
Managing Entities
Traps, etc
Embedded Device Management
For High Availability: Do Not Disturb
Model – No Surprises
- Consistency Between Managers and Device Software
Transactions – No Mistakes
- Always Clean Configuration, Safe Rollback
Validation – Predictable Rejection
- Model Validation plus Validation by Applications
Northbound Mediation Layer – Device Remains Simple
- Normalized Agent Interface: Many Agents; One Southbound API
Dynamic Configuration at Run-time – Always On
- Ability to Change System Without Downtime
EMS, NMS, OSS
The Management Framework Should Scale
B
A
C
D
Operational Data
State
Status
Statistics
Configuration Data
Persistent
Transactions
Ideally it is ACID*
*Atomic, Consistent,
Isolated, Durable
XMLRPC, NETCONF, SNMP
CLI, Web
Notifications
Events
Alarms
Actions
BA C
D D D D
C
D
D
B
A
D
D
D
B
AB
A
C
XML-RPC
SNMP
NETCONF
CLI
Web
CLI
Web
Cloud
Scalable Cloud-Based Clusters with Availability
Middleware Stack
S1 S2 S3
Middleware Stack
S1’ S2’ S3’
2N Redundant
MW Stack
A1 B1
MW Stack
A2 B2
MW Stack
A3 B3
MW Stack
A1’
A2’
A3’
B1’
B2’
B3’
N+M (3+1) Redundant
MW
C1
MW
C2
MW
C3
MW
C4
MW
C9…
N-Way Active
MW
C1
C2
C3
MW
C4
C5
C6
MW
C7
C8
C9
N-Way Active
CLI
Web
XML-RPC
NETCONF
SNMP
Direct UserEMS, NMS, OSS
…
Elastic
Scale
Embedded Device Management• Model Driven Object Manager
• Model-Driven Northbound Agents
• ACID Configuration Datastore Transactions w/Rollback
• Fully Distributed, Unified Application Interface
CLI
Web
XML-RPC
Technician Interface:
Development
Maintenance
Managing Entities
Use Cases: Single Node Systems
Middleware Stack
S1 S2 S3S3
CLI
Web
XML-RPC
NETCONF
SNMP
Direct UserEMS, NMS, OSSEmbedded Device Management• Model Driven Object Manager
• Model-Driven Northbound Agents
• ACID Configuration Datastore Transactions w/Rollback
• Fully Distributed, Unified Application Interface
CLI
Web
XML-RPC
Technician Interface:
Development
Maintenance
Managing Entities
A B C
Element Embedded Management
CLI
Web EMS/NMS/OSS
Element EM
SN
MP
NE
TC
ON
F
XM
L-R
PC
CLI
WE
B
System Model in YANG
Northbound AgentsModel-Driven
Configuration DatastoreACID Transactions with Rollback
Redundant in HA Deployments
Simple Southbound APIDynamic and Fully Distributed
submodule ethernet-config
{
belongs-to element { prefix elem; }
include element-types;
include element-base-config;
description “Defines the configuration..”
augment “/configuration/cluster/node”
{
container ports
{
list ethernet
{
key “port-id”;
ACID ACID
Element Embedded Device Management
CLI, Web, SNMP, NETCONF, XML Configuration
Datastore
System
Model(YANG, XML)
Operational Data
transactions
Unified: Enea Element Middleware
AMF High AvailabilitySoftware
Management
ISU
Northbound
Agents
Embedded Management
Object
Manager
Runtime
DebugDebug and Trace
Log
Trace
LINXMessagingName Server
Event Mgr
Flow Control
Check Point
Data
Replication
Data Store
Objects
Technician
User
Interface
Check Point
Data
Replication
Check Point
Data
Replication
Element: A Scalable Application Ready Platform
MSG MSG
EM
Debug
Trace
MSG
EM
HADebug
Trace
Client
1
Client
2
Client
3
Server
B
Server
A
Worker
1
Worker
2
Worker
3
Worker
4
Client
1
Server
A
Worker
1
Worker
2
Worker
1
HAEM
Client
1
Cloud
Solution-Specific Software
Element: Application Ready Platform
SoC ATCA Virtual
Element Middleware
Debug and Trace
Distributed Messaging
Embedded Device Management
CL
I
We
b
XM
L-R
PC
SN
MP
NE
TC
ON
F
CL
I
We
b
XM
L-R
PC
Customer Solution
High Availability
Model-Driven
ManagementDebug and Trace
Dev/Maintenance
Linux
Use Enea Element Middleware to
• Add commodity high-end features• High Availability
• Model-driven Management
• Runtime Troubleshooting
• Scalable Distributed Platform
• Focus on your solution
Enea Element Middleware Scales• From small footprint SoC…
• …to standard ATCA Chassis…
• …to Cloud and Virtual Environments
• Choose the Right Size Platform
Base StationSatellite
LTE
IP TV
M2MOptical
Core
App Server
Media Gateway
Security GatewayFinancial Services
Thank you for Attending
See us at enea.com
Questions?