Lean Drupal Repositories with Composer and Drush

transcript

Lean Drupal Repositories with

Composer and Drush

by Greg Anderson

Photo by Adam Wyles2016

Pantheon.io 2

Session Description

Composer is the industry-standard PHP dependency manager that is now in use in Drupal 8 core. This session will show the current best practices for

using Composer, drupal-composer, drupal-scaffold, Drush, Drupal Console and Drush site-local aliases to streamline your Drupal 7 and Drupal 8 site

repositories for optimal use on teams.

We will answer such gripping questions as:

● How do I avoid placing a copy all of the Drupal core and contrib modules into my repository?

● How do I keep my core and contrib modules up-to-date?

● What if I need to customize .htaccess, or some other file?

● Can I work with a "lean" repository, and still seamlessly use a full repository to deploy?

● How do I share Drush aliases with team members, without using another repository, and without making my alias list too long?

● What is a Drush wrapper script, and how can it help me?

● How does Drupal Console fit in to all of this?

● Should I use test fixtures in my Behat tests, or make a copy of the production site database?

These techniques will also be helpful for solo site developers--you never know, the next person who needs to check out, build and test your Drupal site

from scratch might be you!

Pantheon.io

Then create a Git repository:

cd myproject

git init

git add .

git commit -m “My new project”

Getting Started with Drupal and Composer

One easy step in Composer:

composer create-project drupal-composer/drupal-project myproject --stability=dev

Install and run with Drush:

cd web

drush qd --db-url=mysql://root@localhost/myprojectdb

Answer “yes” when Composer

asks to remove VCS files, or run

with --no-in

teractio

n option

Pantheon.io 4

Preferred Project Structure

webcomposer.json

drupal console

circle.yml

external libraries

behat.yml

behat drush

drupal/core

index.phpcustom-installers

drupal-scaffoldmodules

contrib

custom

vendor

Pantheon.io 5

Specifying Project Layout

"name": "drupal-composer/drupal-project",

"scripts": {

"drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold",

"post-install-cmd": "sh ./scripts/composer/post-install.sh"

"extra": {

"installer-paths": {

"web/core": ["type:drupal-core"],

"web/modules/contrib/{$name}": ["type:drupal-module"],

"web/profiles/contrib/{$name}": ["type:drupal-profile"],

"web/themes/contrib/{$name}": ["type:drupal-theme"],

"drush/contrib/{$name}": ["type:drupal-drush"]

Pantheon.io 6

Renaming Document Root

# Ignore directories generated by Composer

drush/contrib

vendor

docroot/core

docroot/modules/contrib

docroot/themes/contrib

docroot/profiles/contrib

# Ignore Drupal's file directory

docroot/sites/default/files

.gitignore

#!/bin/sh

DOCUMENTROOT=docroot

# Prepare the scaffold files if they are not

if [ ! -f $DOCUMENTROOT/autoload.php ]

composer drupal-scaffold

mkdir -p $DOCUMENTROOT/modules

mkdir -p $DOCUMENTROOT/themes

mkdir -p $DOCUMENTROOT/profiles

post-install.sh

{ "extra": { "installer-paths": { "docroot/core": ["type:drupal-co "docroot/modules/contrib/{$name} "docroot/profiles/contrib/{$name "docroot/themes/contrib/{$name}" "drush/contrib/{$name}": ["type: } }}

composer.json

Pantheon.io 7

Scaffold Files

drupal/core

scaffold files

modules

contrib

custom

.htaccess

autoload.php

index.php

robots.txt

updates.php

examples.sites.php

default

default.services.yml

default.settings.php

Drupal.org

drupal-scaffold

Pantheon.io 8

Custom Scaffold Configuration

"name": "pantheon-systems/drupal-project",

"scripts": {

"extra": {

"drupal-scaffold": {

"source": "https://github.com/pantheon-systems/drops-8/archive/{version}.tar.gz",

"includes": [

"sites/default/settings.php",

"sites/default/settings.pantheon.php"

Pantheon.io 9

Document Root at Project Root

composer.json

drupal console

external libraries

contrib

custom

vendor

circle.yml

private

Privatesubfolder

Pantheon.io 10

composer.json Relocated to Private Directory

"scripts": {

"extra": {

"installer-paths": {

"../core": ["type:drupal-core"],

"../modules/contrib/{$name}": ["type:drupal-module"],

"../profiles/contrib/{$name}": ["type:drupal-profile"],

"../themes/contrib/{$name}": ["type:drupal-theme"],

"../drush/contrib/{$name}": ["type:drupal-drush"]

Pantheon.io 11

Updating Code

drupal/core

scaffold files

modules

contrib

custom

.htaccess

autoload.php

index.php

robots.txt

updates.php

examples.sites.php

default

default.services.yml

default.settings.php

Drupal.org

drupal-scaffold

composer update

composer drupal-scaffoldgit pull

Project FilesOnly

Scaffold FilesOnly

Composer and Scaffold

Pantheon.io 12

Customizing Scaffold Files

"scripts": {

"post-install-cmd": "sh ./scripts/composer/post-install.sh",

"post-drupal-scaffold-cmd": "cat htaccess-append.txt >> ../.htaccess"

To patch non-scaffold files, use:

cweaga

ns/com

poser-

patche

Pantheon.io 13

Checking for Security Updates

With Drush

With Composer

$ drush pm-updatestatusChecking available update data ... [ok]Checking available update data for Drupal. [ok]Checking available update data for Token (token). [ok] Name Installed Version Proposed version Message Token (token) 7.x-1.2 7.x-1.6 SECURITY UPDATE available

$ composer require roave/security-advisories:dev-master$ composer require drupal-composer/drupal-security-advisories:7.x-dev./composer.json has been updatedLoading composer repositories with package informationUpdating dependencies (including require-dev)Your requirements could not be resolved to an installable set of packages.

Problem 1 - drupal-composer/drupal-security-advisories 7.x-dev conflicts with drupal/token[7.1.2]. - Installation request for drupal-composer/drupal-security-advisories 7.x-dev -> satisfiable by drupal-composer/drupal-security-advisories[7.x-dev]. - Installation request for drupal/token 7.1.2 -> satisfiable by drupal/token[7.1.2].

n.b. Requires git_deploy module if using Composer with --prefer-source, or if dev modules are used.

Pantheon.io 14

Manage Custom Modules with Composer

webcomposer.json

drupal console

circle.yml

external libraries

behat.yml

behat drush

drupal/core

contrib

custom

vendor

Variant

Use private modules in multiple projects.

Pantheon.io 15

Let’s Make a Module!

$ drupal generate:module $ drupal generate:controller

Pantheon.io 16

Customize Module’s composer.json

{ "name": "greg-1-anderson/snazzy", "type": "drupal-module", "description": "A Snazzy Module", "keywords": ["Drupal"], "license": "GPL-2.0+", "homepage": "https://github.com/greg-1-anderson/snazzy", "minimum-stability": "dev", "support": { "issues": "https://github.com/greg-1-anderson/snazzy/issues", "source": "https://github.com/greg-1-anderson/snazzy" }, "require": { }}

Pantheon.io 17

Add Our Custom Module to Drupal Project

{ "repositories": [ { "type": "vcs", "url": "https://github.com/greg-1-anderson/snazzy.git" } ], "require": { "composer/installers": "^1.0.20", "drupal-composer/drupal-scaffold": "^1.0", "greg-1-anderson/snazzy": "dev-master" }, "extra": { "installer-paths": { "web/modules/custom/{$name}": ["greg-1-anderson/snazzy"], "web/modules/contrib/{$name}": ["type:drupal-module"] } }}

Pantheon.io 18

Generate require Section from Existing Site

composer create-project drupal-composer/drupal-project myproject --stability=dev

cd myproject

drush composer-generate @remote

Look up the modules and themes

used on @remot

e and add them to

the require section of the

compos

er.jso

n in the cwd.

Pantheon.io 19

Lean Repository Deployment

Request Test

Request Build

Deploy

Pull Lean Repo

Install

Update

Pantheon.io 20

Add a Project to Circle

A couple of clicks will get Circle CI building your project.

There are a few credentials that should also be set up:

● GitHub OAuth Token● Terminus Machine Token● SSH Key Pair

Pantheon.io 21

Generate OAuth Tokens for all Services

GitHub Personal Access Tokens Pantheon Machine Tokens

OAuth tokens are getting to be very common; many services provide them.They work like passwords, but can have limited permissions, and may be revoked.

Pantheon.io 22

Copy OAuth Tokens into CI Envrionment Variables

OAuth Tokens:

Pantheon.io

Prevents ‘composer install’

from failing due to hitting the

GitHub rate limit.

Use OAuth Environment Variables in CI

dependencies: pre: - composer config -g github-oauth.github.com $GITHUB_OAUTH - terminus auth login --machine-token=$PANTHEON_MACHINE_TOKEN

circle.yml

Log in to Terminus via machine token, e.g. to later deploy from

dev to test.

Pantheon.io 24

Create SSH Key Pair Specifically for CI

$ ssh-keygen -C me+mysite-ci@example.com -f my-ci-keyfile

Add Private Key to Circle SSH Permissions Add Public Key to Provider SSH Keys

For linux servers: ssh-copy-id user@isp.com

Pantheon.io 25

Dependency Hell

Bootstrapping Drupal 8 via a global Drush will ONLY WORK if the dependencies of the two projects are in perfect alignment. Upgrading one without upgrading the other is dangerous.

Drush Drupal 8

require autoload.php

Bootstrap require autoload.php

Pantheon.io 26

Simple Example of Dependency Hell

class Sub extends Base{public function foo(){return $this->bar();

class Base{private function bar(){…

class Sub extends Base{public function foo(){return $this->boz();

class Base{private function boz(){…

FancyLib v1.0.1 FancyLib v1.0.2

Sub.php Sub.php

Base.php Base.php

Semantic Versioning will not save

Two autoloaders that contain multiple

copies of the same library can still fail,

even if both conform to the same

public API.

Pantheon.io 27

Dependency Hell Affects Drupal Console Too

Drupal Console, like Drush, keeps dependencies in sync with the Drupal 8 release with the same version number. Problems can still occur if versions become mismatched.

Drupal Console Drupal 8

require autoload.php

Bootstrap require autoload.php

Pantheon.io

Place Drush and Drupal Console in Drupal’s composer.json file:

cd /path/to/drupal-8-root

composer require drush/drush:8.*

composer require drupal/console

Solution is to Use a Single Autoloader

CRITICAL

Pantheon.io 29

Gentoo’s “Composer Problem”

“ As long as the necessary require statements are left in the code (where they belong), we can ignore Composer entirely and install the package with the system package manager. We set PHP's include directory for our users, so require('Class.php'); already looks in the right place. https://wiki.gentoo.org/wiki/Project:PHP/The_Composer_problem

Pantheon.io 30

How Could We Fix Things for Gentoo?

One Class.php shared by every application is not going to work!

Make an autoload.php that loaded versioned Class.php files from a global location?

How would all of the different versions of Class.php be managed?

How would you apply a security update for Class.php?

By the time we have solved all of

these issues, we have pretty much re-

invented Composer.

Pantheon.io

Drush startup now happens in four phases:

Drush Startup

Drush Finder Drush Wrapper Drush Launcher Drush Application

Pantheon.io 32

Drush Finder

Responsible for finding the correct Drush to run

- Checks to see if it can find a Drupal site

- If the Drupal site contains a Drush script, use it

- If no Site-Local Drush is found, use global Drush

Does not consider alias files.

PHP Scriptnamed

“drush”

Pantheon.io 33

Drush Wrapper

Optional. Located at the Drupal Root if it exists.

User may customize this script to:

- Add site-specific options (e.g. commandfile locations)- Turn off global search locations with --local option- Select the location of the Drush launcher (if non-standard)

If there is no Drush Wrapper, then the Drush Finder will find and execute the Drush Launcher.

Shell Scriptnamed

“drush.wrapper”

Pantheon.io 34

Drush Launcher

Sets up the PHP operating environment.

- Select php.ini file to use

- Select php executable to use

- Passes info about environment to Drush

The launcher will always use the Drush application located in the same directory.

Shell Scriptnamed

“drush.launcher”

Pantheon.io 35

Drush Application

Contains all the code that is Drush.

- Load configuration and command file

- Parse site alias files

Might dispatch again, e.g. if site alias is remote.PHP

Applicationnamed

“drush.php”

Pantheon.io 36

Drush Phar

Bundles all of the code from the Drush application into a single file.

- Does not use Drush Launcher (no PHP executable selection).

- Will run a Drush Wrapper script if one is available.

If a Drush Wrapper or site-local Drush is found, they will be executed via redispatch. The site-local Drush may be a Phar or a regular Drush application.

Phar file named

“drush.phar”

Pantheon.io 37

Sharing Drush Aliases

cd "`dirname $0`"

private/vendor/bin/drush.launcher \

--local \

--include=../drush/commands \

--alias-path=../drush/aliases \

--config=../drush/config "$@"

drush.wrapper

aliases

commands

index.php

aliases.drushrc.php

$ cd web

$ drush sa @live

$aliases["live"] = array (

'remote-host' => 'server.isp.com',

'remote-user' => 'www-admin',

'root' => '/srv/www/live.example.com',

'uri' => 'http://example.com',

Pantheon.io 38

Behat Driver Enhancement

cd /path/to/drupal-projectcomposer require drush-ops/behat-drush-endpoint

Adds behat Drush command to your site; used to remotely create content.

Pantheon.io 39

Behat Fixtures

$ drush behat import --file=fixtures.yml

create_term: places: - name: Kingdom of Imaginarium - name: Empire of Fabrication offices: - name: Grand Poohbah - name: Undersecretary of Things - name: Minister of Ministering

Option 1: Use Behat’s built-in “Background”

Option 2: Create fixtures with behat-drush-endpoint

Deletes and re-creates fixtures for every test; useful for testing operations that modify or delete terms.

Taxonomy IDs stay constant for every test run.

ALSO (for Drupal 8): drush site-install --config-dir=export-dir

Pantheon.io 40

That’s a Wrap!

Follow me on Twitter for slides and other Composer + Drush updates:

@greg_1_anderson

Lean Drupal Repositories with Composer and Drush

Technology