bmclibA Baseboard Management Controller library

One library to rule them all?

Fosdem 2019

Juliano MartinezJoel Rebello

A BMC is a system on chip that integrates various computer components in a single die/package,

roughly similar to the Broadcom SoC found on a Raspberry PI,

except, the BMC runs within a server/chassis/switch/JBOD/JBOF with its own OS/Firmware

Baseboard Management Controller

~

● Out of Band access to servers/switches/JBODs, etc

● Last resort to power cycle, reboot, hard reset

● IPMI/VNC/iKVM/Serial console access

● Inventory information

● Hardware logs

● Root of trust

Various flavours, single function

Asset lifecycle management BMCs

Common SoC BMC - AST2400/AST2500

● 400MHz/800MHz ARM CPU

● 512 MB DDR3/DDR2 SDRAM

● PCIe VGA

● 10/100/1000 Mbps NIC (Dedicated/Shared)

● Web, IPMI, SSH (SMASH)

Specs

● IPMI - common across all vendors, although shitty and insecure

● SSH - there's no standards (vendor specific implementations)

● Web interfaces - slow and buggy

● API - none or inconsistent implementations of Redfish[0]○ Redfish is an odata based API, which is overkill[1] for a BMC device○ Unreliable/buggy implementations across vendors○ Every vendor seems to be heading in their own direction○ Promising, maybe in the future this will be better

Standards

What are those?

0. https://www.dmtf.org/standards/redfish1. http://docs.servicestack.net/why-not-odata

@scale

● 50K and growing set of bare metal servers and storage hardware.

● Hardware from multiple vendors, generations.

The hardware

● Four engineers, 50K servers ~ 12500 servers/engineer.

● Treat servers as light bulbs - plug and play.

● Reliably provision.

● Accurately inventorize.

● Manage BMC configuration.

● Diagnose hardware problems.

● All this done using the BMC, without manual intervention, at scale.

@scaleThe challenges

bmclib One library to abstract BMC interaction

https://github.com/bmc-toolbox/bmclib

bmc-toolbox

https://github.com/bmc-toolbox

bmclib

Dora collect inventory

apply configuration

execute actions

BMCs

update firmware

bmcbutler

Actor

bmcbutler

apply configuration

execute actionsbmcbutler

inventory

BMC config

A BMC configuration management tool

go get github.com/bmc-toolbox/bmcbutler

bmclib

BMCsBMCs

server

BMC

server

BMC

server

BMC

server

BMC

Dora

collect inventory

DB

Dora

bmclib

BMCsBMCs

server

BMC

server

BMC

server

BMC

server

BMC

Asset inventory and explorer

go get github.com/bmc-toolbox/dora

ActorConsistent Web API interface to BMCs

go get github.com/bmc-toolbox/actor

execute actions

foremanlazy sysadmin

actor

BMCsBMCs

server

BMC

server

BMC

server

BMC

server

BMC

bmclib in action

BMCsBMCs

bmclibserver

BMC

grab screen preview

Inception model re-trained

300x300 px

Lazy sysadmin

Hey Actor,Get me the the state ofserver-foobar?

bare metal state identification

BMCsBMCs

server

BMC

● bmclib abstracts various vendor BMCs into a single API - so we can focus on building tools to manage them

● If you are looking to inventorize, configure, update server BMCs, check out the bmc-toolbox

● Asking what you need and get what you expect for orchestration is a must

● BMCs are a fundamental part of a servers lifecycle, it's time they got more attention

● You can help! create an issue/PR - If you work with bare-metal servers and would like support for your BMC

● Avoid vendor lock-in and have defined requirements to require

https://github.com/bmc-toolbox

Takeaways