Backporting the kernel with SmPL

@mcgrofmcgrof@suse.com

Luis R. Rodriguezhttp://www.do-not-panic.com

Slides: CC BY-SA Luis R. Rodriguez | Image: CC BY-SA Torkild Retvedt

The Linux kernel backports project

Slides: CC BY-SA Luis R. Rodriguez | Image: CC BY-SA Torkild Retvedt

● https://backports.wiki.kernel.org● git://git.kernel.org/pub/scm/linux/kernel/git/backports/backports.git● Started in 2007 - folded under the Linux Foundation backports working group● We strive to backport the Linux kernel automatically● Stable releases, and linux-next daily releases (well sort of)● Well over 800 drivers now, we stopped counting● Had to make compromises recently to scale:

○ dropped DRM drivers, might drop some others soon, only carry things folks need

○ Only support kernels >= 3.0 (match kernel.org)○ Carrot: get upstream, use backports for releases.○ Proprietary drivers cannot use this, this is for upstream drivers

● irc.freenode.net #kernel-backports● 3 core developers, 2 co-maintainers, Hauke Mehrtens now doing most of the work

○ Ethernet, Wireless, Bluetooth, NFC, ieee802154, Media, Regulator

A slightly different use case

Slides: CC BY-SA Luis R. Rodriguez | Image: CC BY-SA Torkild Retvedt

● Coccinelle engine developed to help evolve the Linux kernel, INRIA / IRILL help evolve and maintain it, you may have seen patches from Julia Lawall, Peter Senna Tschudin, and now others

● Collateral evolution concept● Linux kernel developers might use Coccinelle once / twice a

month to help create collateral evolution on a series of device drivers

● On backports we need to use it daily and against all drivers we carry, well over 800 drivers now, and use it for every single SmPL patch we have

The old way

Image: CC BY-NC-SA ecololo

A slightly better way

Image: CC BY-SA Teza Harinaivo Ramiandrisoa

The Coccinelle SmPL way

Image: CC BY-NC Sergiu Bacioiu

@@struct net_device *dev;struct net_device_ops ops;@@-dev->netdev_ops = &ops;+netdev_attach_ops(dev, &ops);

How about complex stuff? Threaded IRQ?

Image: CC-BY m4tik

--- a/drivers/net/wireless/b43/main.c+++ b/drivers/net/wireless/b43/main.c@@ -4290,9 +4299,17 @@ static int b43_wireless_core_start(struc

goto out; }

} else {+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,31)

err = request_threaded_irq(dev->dev->irq, b43_interrupt_handler,

b43_interrupt_thread_handler, IRQF_SHARED, KBUILD_MODNAME, dev);

+#else+ err = compat_request_threaded_irq(&dev->irq_compat,+ dev->dev->irq,+ b43_interrupt_handler,+ b43_interrupt_thread_handler,+ IRQF_SHARED, KBUILD_MODNAME, dev);+#endif

if (err) { b43err(dev->wl, "Cannot request IRQ-%d\n", dev->dev->irq);

Backporting threaded IRQ with SmPL@ threaded_irq @identifier ret;expression irq, irq_handler, irq_thread_handler, flags, name;type T;T *private;@@

+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,31)ret = request_threaded_irq(irq,

irq_handler, irq_thread_handler, flags, name, private);

+#else+ret = compat_request_threaded_irq(&private->irq_compat,+ irq,+ irq_handler,+ irq_thread_handler,+ flags,+ name,+ private);+#endif

The data structure type was inferred by the Coccinelle engine !

And we get to modify it!

@ modify_private_header depends on threaded_irq @type threaded_irq.T;@@

T {+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,31)+ struct compat_threaded_irq irq_compat;+#endif...};

It gets better● Coccinelle revealed

inconsistencies on the backport, the new struct compat_thread_irq was pegged on different data structures on different drivers

● Coccinelle revealed two collateral evolutions were used on that patch

● Amount of time to generate backport reduced, even though it automatically backported this collateral evolution to 10 new drivers ! How and why?

Image: CC BY-SA Teza Harinaivo Ramiandrisoa

Does it scale? Took work, but yes!

● Run on big iron server donated by HP, SUSE, Linux Foundation: 32 cores, 236 GiB RAM

● /pub/mem tmpfs● Generation of code: all in RAM● Compilation tests with ckmake: all in RAM● At first all SmPL patches were concatenated together

and Coccinelle run only once, not parallelized. On uniprocessor runs / unparalleled environments it worked best

● Coccinelle script to parallelize○ requires running Coccinelle a few times, specify

the bucket, written in shell

Parallelizing coccinelle shell script

#!/bin/bash# By Kees Cook# http://comments.gmane.org/gmane.comp.version-control.coccinelle/680set -eMAX=$(getconf _NPROCESSORS_ONLN )dir=$(mktemp -d)for i in $(seq 0 $(( MAX - 1 )) ); do

spatch -max $MAX -index $i -very_quiet "$@" > $dir/$i.out &donewaitcat $dir/*.outrm -f $dir/*.outrmdir $dir

Parallelizing improvements

Helping Coccinelle: needle in the haystack

Image: CC BY-NC Sergiu Bacioiu

@ module_pci @declarer name MODULE_DEVICE_TABLE;identifier pci_ids;@@

MODULE_DEVICE_TABLE(pci, pci_ids);

@ simple_dev_pm depends on module_pci @identifier ops, pci_suspend, pci_resume;declarer name SIMPLE_DEV_PM_OPS;declarer name compat_pci_suspend;declarer name compat_pci_resume;@@+compat_pci_suspend(pci_suspend);+compat_pci_resume(pci_resume);SIMPLE_DEV_PM_OPS(ops, pci_suspend, pci_resume);

@@identifier backport_driver;expression pm_ops;fresh identifier backports_pci_suspend = simple_dev_pm.pci_suspend ## "_compat";fresh identifier backports_pci_resume = simple_dev_pm.pci_resume ## "_compat";@@

struct pci_driver backport_driver = {+#if (LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,29))

.driver.pm = pm_ops,+#elif defined(CONFIG_PM_SLEEP)+ .suspend = backports_pci_suspend,+ .resume = backports_pci_resume,+#endif};

Security folks might like this

Image: CC-BY m4tik

#include

struct net_device_ops {};

struct net_device {struct net_device_ops *netdev_ops;

};

struct bubble_ops {};

struct bubbles {struct bubble_ops *netdev_ops;

};

static struct net_device_ops my_netdev_ops = {};

static struct bubble_ops my_bubble_ops = {};

static struct parent {struct net_device *dev;int b;

};

static struct parent_usb {struct net_device *net;int b;

};

int main(void){

struct parent *p = malloc(sizeof(struct parent));struct parent_usb *p_usb = malloc(sizeof(struct parent));struct net_device *dev = malloc(sizeof(struct net_device));struct bubbles *bubble = malloc(sizeof(struct bubbles));

dev->netdev_ops = &my_netdev_ops;bubble->netdev_ops = &my_bubble_ops;

free(dev);free(bubble);free(p);free(p_usb);

p->dev = dev;p->dev->netdev_ops = &my_netdev_ops;p_usb->net->netdev_ops = &my_netdev_ops;

return 0;}

@@expression dev;expression ops;@@-dev->netdev_ops = ops;+netdev_attach_ops(dev, ops);

@@struct net_device *dev;struct net_device_ops ops;@@-dev->netdev_ops = &ops;+netdev_attach_ops(dev, &ops);

https://github.com/mcgrof/netdev-ops.git

1. make test1

2. git checkout -f

3. make test2

Conclusions

Image: CC-BY m4tik

● We removed support for kernels older than 3.0, so only 5 SmPL patches on backports now

● Overkill for small patches● Use SmPL for the hard ports, to backport real

collateral evolutions● Must break down every patch into separate atomic

pieces● First set of changes submitted upstream to help

with making backporting easier. Example: wrapper for static inlines for data structures

● Thanks INRIA, IRILL !

Further reading

Image: CC-BY m4tik

● Coccinelle post with a bit more elaborate details● Backports wiki - https://backports.wiki.kernel.org/

Questions?

Image: CC-BY m4tik