SlideShare a Scribd company logo
© Ibuildings 2014/2015 - All rights reserved
#DrupalDaysEU
DRUPAL
CONTINUOUS INTEGRATION
by Claudio Beatrice
#DrupalDaysEU
© Ibuildings 2014/2015 - All rights reserved
Gold Sponsors
#DrupalDaysEU
© Ibuildings 2014/2015 - All rights reserved
Media Sponsors
Silver Sponsors
DRUPAL
CONTINUOUS
INTEGRATION
Introduction
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Continuous Integration
“Continuous Integration” is a
software development practice
where members of a team
integrate work frequently,
usually each person integrates at
least daily - leading to multiple
integrations per day. Each
integration is verified by an
automated build (including
tests) to detect integration
errors as quickly as possible.
— Martin Fowler
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Continuous Integration
• automating repetitive tasks such as build, QA and
deploy
• fast and frequent integrations
• fast feedback loop, catch errors asap
• ability to deploy quickly and keep track of every
released set of code
ADVANTAGES
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014-2015http://tenwarp.com/
Continuous Integration
developer
commits triggers
config
static
analysis
unit /
functional
tests
runs
runs
runs
report
FAIL
deploy
smoke /
functional
tests
report
SUCCESS
SUCCESS
FAIL
CI
server
revision
control
log
logs
logs
logs
BUILD PROCESS EXAMPLE
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Continuous Integration
TOOLS
• SCM: git, svn, cvs, …
• CI SERVER: jenkins
• BUILD TOOL: phing
• DRUPAL AUTOMATION: drush
• DEPENDENCIES MANAGEMENT: drush make & composer
• DRUPAL CDD TOOLS: features & migrate
• TESTING: PHPUnit & Behat
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Continuous Integration
DRUPAL - I
Drupal, as opposed to “classic” frameworks such as
Symfony or Zend, does not make the setup and
maintenance a Continuous Integration cycle easy
due to the high coupling between data and
configuration within its database.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Continuous Integration
DRUPAL - II
To overcome this problem it’s necessary to adopt a
strictly controlled, disciplined work methodology
and to make use of a set of tools that allow us to
transform configuration into code, to easily deal
with content and to incrementally update
database’s schema.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Continuous Integration
WORKFLOW - I
A developer’s workflow consist not only in the plain
old features implementation, but also in the correct,
precise and complete export of all the element of
configuration that are necessary to rebuild the
functionalities on several environments.
!
Key modules: Features and Strongarm.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Continuous Integration
WORKFLOW - II
It’s often necessary to setup some default content that
may vary during development and that might be
expanded or updated once the site has been
published. It’s therefore necessary to formalise in code
all the data import procedures in order to better
administrate them and to be able to replicate them at
any given time.
!
Key module: Migrate.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Continuous Integration
WORKFLOW - III
As soon as a self-contained task such a bug-fix or a
new feature implementation has been completed,
it’s necessary to commit and push that unit of work
to the central repository in order to integrate, test
and deploy it automatically on the development
environment.
!
Key tools: git/svn.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Continuous Integration
WORKFLOW - IV
!
Writing automated tests is necessary in order to
guarantee the behaviour of a given implementation
and to make sure that future changes won’t break
the work already done, creating regressions and
dealing potential damage to users and clients.
!
Key tools: PHPUnit, Behat.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Q & A
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Continuous Integration
EXERCISES
1. What are the main advantages of a Continuous Integration system?
2. When do Unit Tests are executed during the build? How about
functional? Why are they executed at different times?
3. Why is it important to write tests?
4. Which benefits come from using a Source Control Management system?
5. Why is Drupal so hard -compared to other tools such as “classic”
frameworks- to use in a Continuous Integration environment?
6. What tools allow Drupal to be successfully employed in a Continuous
Integration development cycle and for what reasons?
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
DRUPAL
CONTINUOUS
INTEGRATION
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Phing is a build system written in PHP that allows
developers to describe the automated operations
used during development, deployment and
maintenance using plain XML files.
Phing
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
• Simplicity: it’s based on well-known, declarative XML syntax
and features a very rich documentation
• Power: thanks to the big amount of available tasks, it is able
to perform almost anything you could do using a
command line interpreter
• Extensibility: as it is written in PHP, it’s easy to extend and
debug
• Automation: the whole application configuration and
deployment process can be handled by build files
Phing
BENEFITS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
• execute tests and log their results
• perform filesystem operations
• interact with source configuration management systems
• generate documentation
• transfer files from/to remote systems
• fire drush commands
• invoke shell commands
Phing
FEATURES
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
There are two available file types in Phing to configure
the build: .properties and .xml files. The former
contains a list of key/value variables, while the latter
contains the “instructions” to execute.
In a usual setup there are up to five different
environments: loc, dev, stage, debug and prod. Each
of these ones will be configured using a
dedicated .properties file in order to make the build
file environment-agnostic so to reuse most of its code.
Phing
MAIN FILES AND ENVIRONMENTS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Phing
PROPERTY FILE
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Phing
BUILD FILE - PROJECT TAG
For every build file, there must be a single, unique,
root “project" tag. Within that "target", "task" and
“type" tags can be used.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Phing
BUILD FILE - TYPE TAG
Besides default types such as numbers or strings, it is
possible to define more complex, nested types. An
interesting feature of Phing’s types is the ability to be
referenced by id later.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Phing
BUILD FILE - TARGET TAG
“Targets” are collections of components that are
assigned a unique name within the project. It is possible
to invoke other targets within a target and it’s also
possible to set dependencies between different targets.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Phing
BUILD FILE - TASK TAG
“Task”s are responsible for the execution of specific units
of work and are usually configurable through attributes
and nested tags. A few examples of what tasks do could
be: files copying, archive compression or drush
commands execution.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Q & A
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Phing
EXERCISES
1. What are the main benefits of using a build system?
2. Why is Phing more convenient than a bunch of shell
scripts?
3. What files does Phing use to work? What’s their
use?
4. What are the fundamental elements of a build file?
What’s their use?
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
DRUPAL
CONTINUOUS
INTEGRATION
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
Drush is a command-line shell and scripting
interface for Drupal, a veritable Swiss Army knife
designed to make life easier for those who spend
their working hours hacking away at the command
prompt.
— Drush Team
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
• Drupal Power: ability to simply, swiftly and
programmatically execute Drupal-specific tasks
• Integration: working with other command-line
tools is very easy and effective
BENEFITS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
• code and database backup, restore e sync
• cron tasks execution
• automatic installations
• variables, cache and logs CRUD operations
• modules and themes administration
• SQL commands execution
• users administration
• dependency management
• remote tasks execution
FEATURES
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
Drush make is a Drush extension that deals with
modules, themes, libraries and patches
management. It makes use of a manifest file to index
and download the required versions of the packages
used to build a Drupal website.
MAKE - I
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
MAKE - II
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
Drush Make allows developers to specify a
destination sub-folder using the “subdir” option.
We’ll leverage that to establish the convention of
having four sub-folders for modules: contrib,
custom, devel and features. This division will help in
keeping a cleaner division between the modules
and will save us the burden of versioning “contrib”
modules.
MAKE - III
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
Drush allows the developers to create a kind of
labels called Site Aliases: these are nothing but
configuration groups that fall under an arbitrary
name and make the execution of some Drush
commands easier. In our case those aliases match
the environments defined for our project, and will
look something like myproject.loc and
myproject.dev.
SITE ALIAS - I
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
SITE ALIAS - II
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
Drush looks for Aliases definitions in several locations:
1. in the path set in the “alias path” variable of the options array
found in the drushrc.php file
2. in the path specified in the —alias-path command-line option
3. in one of the following folders: “/etc/drush”, “$HOME/.drush” or
the current site’s “sites/all/drush”
In order to keep Drush Aliases clean and separated, we’ll use our
projects’ “sites/all/drush” as the go-to location to store their
configuration so not to pollute the global configuration.
SITE ALIAS - III
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
Drush implements two extremely useful commands
to keep files and databases in sync between hosts:
rsync e sql-sync.
• drush rsync @mysite.loc @mysite.dev
• drush sql-sync @mysite.loc @mysite.dev
SYNCHRONISATION
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Q & A
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Drush
EXERCISES
1. What’s Drush?
2. Why is it convenient to use it?
3. What are its most important features?
4. What’s the use for a make file?
5. What’s a Drush Site Alias? What’s its use?
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
DRUPAL
CONTINUOUS
INTEGRATION
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Jenkins
Jenkins is a Java application useful to manage the
execution of those repetitive tasks that are needed
to build a software project.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Jenkins
• automation of deployments procedures
• speed-up of repetitive tasks such as test runs or the
creation of build artifacts to deploy on our projects’
target environments
• logging and monitoring for a handful of code-related
stats such as test results or code quality analysis
• convenient web ui to configure all the needed jobs
BENEFITS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Jenkins
FEATURES
• integration with several source configuration management
systems
• tests results and code quality reports generation
• integration with several build systems
• jobs status notifications
• configurable web ui
• hundreds of plugins
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Jenkins
INSTALLATION
Jenkins
PLUGINS
Jenkins
JOB CREATION - I
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Jenkins
JOB CREATION - II
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Jenkins
DASHBOARD
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Q & A
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Jenkins
ESERCIZI
1. What’s Jenkins?
2. What are its tasks?
3. What’s a Job?
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
DRUPAL
CONTINUOUS
INTEGRATION
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
The Feature module is able to export parts of the
configuration found in the database into “ordinary”
Drupal modules. The ability to produce a set of files
containing a specific functionality starting from the
database makes them much easier to manage and
deploy across several systems and installations.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
• Removes the need for manual, repetitive work that’s necessary to
reproduce all of a website’s features and traits by encapsulating
them in portable, standard Drupal modules;
• Exports directly to Drupal modules or, in other words, text files:
that means it’s easy to version them using a source control
management system, which in turn makes easy to keep the history
of changes and allows multiple people to work on the project
concurrently;
• Allows to transport functionalities across websites in an easy,
reproducible and automatic fashion.
Features
BENEFITS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
Local Stage
Dev
Prod
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
CREATION - I
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
CREATION - II
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
ADMINISTRATION
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
OPERATIONS - ENABLING A FEATURE
Enabling a Features-exported module executes the
same operations a “normal” module executes, such as:
• Schema installation;
• System Register and Cache update;
• Execution of hook_install();
• Execution of hook_enable().
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
OPERATIONS - UPDATE & REVERT
Features Update: updating a feature
consists in the act of saving some of
the information found in the database
into a specific Feature Module.
Features Revert: reverting a
feature consists in the act of
writing the information found in a
Feature Module into the database.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
features-components (fc) List features components.
features-diff (fd) Show the difference between the default
and overridden state of a feature.
features-export (fe) Export a feature from your site into a
module.
features-list (fl) List all the available features for your
site.
features-revert (fr) Revert a feature module on your site.
features-revert-all (fra) Revert all enabled feature module on
your site.
features-update (fu) Update a feature module on your site.
features-update-all (fua) Update all feature modules on your site.
DRUSH COMMANDS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
• Thanks to the power and flexibility granted by the
Entity API it’s possible to easily create exportable
Entities that are automatically picked up and
handled by Features;
• A lot of contrib modules are already using such
possibility in order to provide exportable
configuration. Some examples could be the Rules,
Views or Context modules.
BEYOND THE DEFAULT OPTIONS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
• Create and maintain “vertical” features such as blog, forum and front page. Avoid
creating Feature Modules that group “exportables” by type, such as “all site’s views” or
“all site’s rules”;
• Do always use English and specific namespaces when naming components
(field_blog_topics instead of field_argomenti);
• Do not use abbreviations unless it’s strictly ncssry.
• Keep a standard format when naming fields: try not to mix casing, dashing,
underscores and so on;
• Define conventions. The Kit Project, although abandoned, is a good source of
inspiration;
• Strive to keep things clean and in order: try to always use Features for importing and
exporting all the developed functionalities.
BEST PRACTICES
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Q & A
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Features
EXERCISES
1. What are the main goals of the Features module?
2. What components can it act upon?
3. What does feature-revert do?
4. What benefits does Features bring?
5. What’s Strongarm used for?
6. Can you name a few modules that allow their
components to be exported by Features?
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
DRUPAL
CONTINUOUS
INTEGRATION
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
The Migrate module makes a powerful and flexible
object oriented framework for importing contents
into Drupal available to developers: it offers out-of-
the-box support for a handful of sources, such as
databases and file formats, and destinations, such as
nodes and taxonomy terms.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
At macro-level, there’s four interesting areas to consider when
working with Migrate:
• An API for defining and managing the migration process;
• A set of Drush commands for managing the migrations, such as
import, rollback and status;
• An Admin UI featuring some of the features exposed by Drush;
• Some example modules that demonstrate a handful of
techniques used to define custom migrations.
FEATURES
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
MigrateSource
id
userName
email
address
MigrateMap
sourceId
destinationId
MigrateDestination
uid
name
mail
UnmigratedDestination
address
ARCHITECTURE
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
SOURCE
• Inherits from the abstract class
MigrateSource
• Provides an interface to the data set
• Iterates on the source’s rows
• Provides a list of fields
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
DESTINATION
• Inherits from the abstract class MigrateDestination
(and usually from the more specific
MigrateDestinationEntity)
• Responsible for persisting data in a specific
Drupal’s content type such as user, node, comment,
file.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
MAPPING TABLE
• Inherits from MigrateSQLMap, which in
turn inherits from the abstract class
MigrateMap
• Describes the key formats for the source
and the destination
• Stores all the information that’s used to
import and rollback each and every
single record
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
FIELDS MAPPING
• Allows to link each destination field with a source
field
• Allows to transform data during the migration to
fix or adapt to the destination requirements.
• Allows to exclude some fields from the migration
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
Handlers are classes that allow to execute additional
operations during the import process.
They are usually used to support migrations where
some specific, non-standard data is connected to
Core Entities: most of the time this will be related to
custom field types.
HANDLER
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
There’s two kinds of Handler: Destination and Field.
The former is used to manipulate the Destination
Entity around saving (pre- and post- save) and to
declare additional fields exposed by the Destination
Entity.
The latter works much like the Destination Handler,
except its methods are invoked for each Field (as
opposed to each Entity).
HANDLER
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
Sometimes there are pieces of content that hold
circular references to each other: for instance a
tutorial post “part 1” could contain a reference to the
“part 2” and vice-versa.
STUB
NODE
nid 1
title tutorial part I
body …
series_nids 2,3,4
NODE
nid 2
title tutorial part II
body …
series_nids 1,3,4
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
During the import process the first node won’t be
able to correctly link to the second as the (new) id is
still unknown: Stubs solve this problem by creating
and linking a temporary, empty node that only holds
an id. As soon as the import process gets to the
stubbed node, it will populate it with all the due
information.
STUB
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
Most of the time migrations are defined as static as that
source and the destination match one-to-one: one row,
one migration.
However, sometimes it’s necessary to “join” multiple
sources having the same structure into a single
destination: this kind of migration is defined as dynamic.
An example could be a migration where you need to
import users from several Drupal sites into a new one.
DYNAMIC MIGRATIONS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
When using this functionality, Drupal will take care of
instantiating each migration multiple times. Besides that, a
dynamic migration differs from a static migration in the
following ways:
• its constructors accepts an array of arguments, where each
of its keys identifies an instance of the migration class;
• it overrides the generateMachineName() method in order
to generate a distinct machine name for each instance, so
to be able to identify them.
DYNAMIC MIGRATIONS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
migrate-analyze (maz) Analyze the source fields for a migration.
migrate-audit (ma) View information on problems in a migration.
migrate-auto-register (mar) Register any newly-defined migration classes.
migrate-deregister Remove all tracking of a migration.
migrate-fields-destination (mfd) List the fields available for mapping in a destination.
migrate-fields-source (mfs) List the fields available for mapping from a source.
migrate-import (mi) Perform one or more migration processes.
migrate-mappings View information on all field mappings in a migration.
migrate-messages (mmsg) View any messages associated with a migration.
migrate-reset-status (mrs) Reset a active migration's status to idle.
migrate-rollback (mr) Roll back the destination objects from a given migration.
migrate-status (ms) List all migrations with current status.
migrate-stop (mst) Stop an active migration operation.
migrate-wipe (mw) Delete all nodes from specified content types.
DRUSH COMMANDS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
ADMINISTRATION
http://tenwarp.com/
Migrate
• Migrate D2D - set of utilities for migrating content from
other versions of Drupal (5, 6, 7) to Drupal 7;
• Migrate Extras - integrates several contrib modules
such as Rules or Pathauto;
• Commerce Migrate - Integrates Drupal Commerce’s
Entities;
• Wordpress/Typo3/PHPBB Migrate - set of utilities to aid
the import of content from other well-known platforms.
BEYOND THE DEFAULT OPTIONS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Q & A
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Migrate
EXERCISES
1. What’s the main goal of the Migrate module?
2. What could be the sources for a migration?
3. What’s the use for the mapping table?
4. What does migrate-rollback do?
5. When is appropriate to use Migrate?
6. What’s a dynamic migration? How is it used?
7. What’s a stub?
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
DRUPAL
CONTINUOUS
INTEGRATION
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
Behat is a testing framework written in PHP with the
purpose of aiding the writing of Acceptance Tests
using Behaviour Driven Development practices.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
IT’S NOT A FUNCTIONAL
TESTING FRAMEWORK
It’s a tool to verify that a given
system satisfies the business
needs of the client.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
Tests are written using a semi-formal language
called Gherkin.
In a nutshell, Behat maps Gherkin sentences -usually
written in English, but it could be any language- to a
class method using to regular expressions.
GHERKIN
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
GHERKIN FEATURE
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
StoryBDD
Behaviour Driven Development based on User
Stories -aka StoryBDD- is a practice that helps into
bringing a development team’s understanding of
business requirements to the same level of the
client’s.
It does so by setting a formal process for asking
questions about requirements and then expressing
the outcome in a specific story-format.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
For each new feature, a developer should answer
the following business questions:
• Who will use it?
• What’s its purpose?
• How is that achieved?
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
StoryBDD
Behat
“I want users to access the site through a dedicated
form”
• Who will use it? a user an anonymous visitor
• What’s the purpose? it’s not stated
• How is that achieved? by using the login form
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
StoryBDD
“I want an anonymous visitor to be able to access the
site using the login form in order to access her
dashboard containing her aggregated data”
!
In order to see aggregated data on my dashboard
as a site visitor
I need to be able to login on the site
Behat
benefit
role
condition
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
StoryBDD
Once a feature is defined, you should start writing all
the scenarios that describe it, using the Context-
Action-Outcome pattern:
!
Given I am in on the homepage
When I login
Then I should see “Welcome Back!”
And I should be on my dashboard
Behat
SCENARIOS
context
action
outcome
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
BEST PRACTICES
And I should see “message” inside “.notifications > p:first-child”
!
It’s recommended to avoid technical details in your scenarios as they
increase the coupling between tests and code and make the stories
harder for non-technical readers -such as the client- to understand.
!
And I should see “message” inside notification area
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
MINK
Each test gets executed within a context where the
methods invoked by the expressions used in the scenarios
can access to a browser-abstraction API called Mink.
There are several drivers available for Mink such as Goutte,
a headless browser written in PHP that’s able to parse
HTML and XML and Selenium, a full-fledged framework
able to programmatically control real browsers.
Oh and there’s a Drush Driver, too! :)
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
MINK CONTEXT
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
MINK BEST PRACTICES
The Page object pattern is a way of keeping context files clean
by separating UI knowledge from the actions and assertions.
In other words, page objects hide the UI and expose clean
services that can be used in the context classes: such
abstraction makes context classes much more clear and helps
avoiding duplication and improving maintainability.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
MINK DRUSH DRIVER
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
EXECUTION EXAMPLE
Gherkin
Feature
Behat Mink
Goutte
Selenium Firefox
Test Results
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Q & A
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Behat
EXERCISES
1. What’s Behat and what’s its main goal?
2. What’s StoryBDD?
3. What’s a scenario?
4. What languages can be used to write the tests?
5. What questions should be answered by a feature description?
6. What are Mink’s main tasks?
7. What browsers can be used run the tests? What are the main
differences between them?
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
DRUPAL
CONTINUOUS
INTEGRATION
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
PHPUnit is a unit testing framework, it’s part of xUnit
family and it has become the de-facto PHP standard
over the years.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
In software engineering, unit testing is a method by
which individual units of code are tested to
determine if they are fit for use.
A unit is defined as the smallest part of an
application that can be tested in isolation: most of
the time that unit is a class or a method.
UNIT TESTING
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
A SUT -an acronym for System Under Test- is defined
as that subset of the software being developed that’s
exercised by a test. It can be a specific method, and
object or even a set of objects.
DEFINITIONS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
Writing automated tests (not necessarily unit tests only) sets some
goals:
• Improving the quality of the product and its code;
• Increasing developers’s understanding of the SUT;
• Decreasing the damage caused by unexpected bugs;
• Simplifying the creation, maintenance and execution of the tests;
• Documenting the behaviours of the applications’ components.
GOALS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
TYPES OF TESTS
Unit
Integration
Functional
Functional: they operate on the
system from the outside, often by
simulating/employing a browser.
Integration: they check how
multiple components of the
system cooperate when
exercised together.
Unit: they make sure that the
single units of code behave as
expects when exercised in
isolation.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
Should I test before or after development?
Traditionally tests -being them unitary or functional- have always
been executed once the implementation was completed.
Such approach has been upturned by the rise of agile
methodologies, where the emphasis is put on the ability to
respond to change quickly rather than sticking to a plan.
Writing tests first makes code inherently testable and therefore
safer to modify and usually produces less verbose and over-
engineered implementations.
METHODOLOGIES
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
Should I test outside-in or inside-out?
As soon as development begins, a very important
decision has to be made: should it start from the big
picture and “zoom in” or from one or some of the
details and “zoom out” in order to compose the
macro-functionality at the end?
METHODOLOGIES
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
Should I test outside-in or inside-out?
By starting development(and testing) from the outside, the
developer forces himself to think as a client first, a method that
helps in making requirements and (temporarily missing)
dependencies more explicit.
Developing “from the inside” on the other hand frees the
developer from the problem of not existing dependencies at
the cost of guessing what’s going to be the right set of things to
implement before reaching the requested level of functionality.
METHODOLOGIES
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
Testing status or behaviour?
There are two main ways to verify the correctness of a SUT:
• Testing the state: it’s often sufficient to verify that given an initial
state, once the system is exercised the ending state then
matches the expected one.
• Testing the behaviour: it’s sometimes necessary to look inside
the black box of the SUT in order to check how the system works
during the execution of the tests to make sure all the behavioural
expectations were satisfied.
METHODOLOGIES
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
TOOLS
PHPUnit provides a number of tools and helpers to aid testing, such as:
• Assertions
• Fixtures
• Database testing helpers
• Stub and mock objects
• Code coverage analysis tools
• Several logging formats
• Integration with several functional testing tools such as Selenium
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
EXAMPLE
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
PHPUnit features about 80 assertions (of which 40 are
“straight” and 40 are “negated”). It’s up to the developer
to pick the best one for the test in order to have the most
expressive tests and the best error messages possible.
ASSERTIONS
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
FIXTURES
When writing tests, one of the most tedious and time
consuming parts is the preparation of the environment
status before the actual execution happens: such status is
known as the fixture of the test .
PHPUnit features quite a big set of methods used to aid in
the creation and sharing of these states, such as:
setUpBeforeClass(), setUp(), assertPreConditions(),
assertPostConditions(), tearDown(), tearDownAfterClass()
and onNotSuccessfulTest(Exception $e).
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
DATA PROVIDERS
It can be useful to be able to run
multiple iterations of the same
test on a larger-than-one data
set. Data Providers allows for a
convenient way to do just that.
Such providers are nothing but
a method that returns an array
containing all the combinations
of arguments a tests will be
called with.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
TEST DOUBLES
It could be helpful -or necessary- to replace a SUT’s
dependencies with so-called “test doubles”, a kind of
objects that looks like the dependency but it’s
created with the sole purpose of satisfying testing
needs.
Such a replacement allows to isolate the SUT and
manipulate its inputs as needed, making the test
more effective and focused on the unit alone.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
TEST DOUBLES
There are various kinds of Test Doubles, each serving a specific
purpose. Meszaros and Fowler suggested the following five
definitions:
• Dummy objects are passed around but never actually used.
Usually they are just used to fill parameter lists;
• Fake objects actually have working implementations, but
usually take some shortcut which makes them not suitable
for production (an InMemoryTestDatabase is a good
example);
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
TEST DOUBLES
• Stubs provide canned answers to calls made during the test, usually
not responding at all to anything outside what's programmed in for
the test;
• Spies are stubs that also record some information based on how they
were called. One form of this might be an email service that records
how many messages it was sent;
• Mocks are pre-programmed with expectations which form a
specification of the calls they are expected to receive. They can throw
an exception if they receive a call they don't expect and are checked
during verification to ensure they got all the calls they were expecting.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
TEST DOUBLES
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
OPTIONS
PHPUnit features a long list of options to configure its behaviour that’s
accessible both from command line and from a configuration file.
The former is a convenient way to quickly get started and try thing out,
but on the long run the phpunit.xml file is probably a more convenient
choice to configure the tool.
It’s a good practice to commit a phpunit.xml.dist file and (git) ignore the
phpunit.xml one: by doing such thing the other developers will be able
to create a phpunit.xml file tailored to their own needs.
PHPUnit will automatically use phpunit.xml if present or phpunit.xml.dist if
not.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
OPTIONS BEST PRACTICES
It’s a good practice to commit a phpunit.xml.dist file
and (git) ignore the phpunit.xml one: by doing such
thing the other developers will be able to create a
phpunit.xml file tailored to their own needs, leaving
the phpunit.xml.dist targeted at automatic builds,
fore example.
PHPUnit is smart enough to automatically use
phpunit.xml if present or phpunit.xml.dist if not.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
OPTIONS
PHPUnit
BEYOND THE DEFAULT OPTIONS
There are more tools provided by the community than what’s implemented
by PHPUnit alone. Here’s a short list of some relevant ones:
• Faker - a library that generates fake-yet-plausible data for a handful of
typical use cases such as names, addresses, numbers, dates, images, etc…
• Mockery - a powerful mocks creation library;
• Prophecy - a new, strongly-opinionated mocks creation library;
• PHP-VCR - a library to record and replay HTTP calls in tests;
• TuTu - a simple and flexible HTTP mocking tool.
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
Q & A
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
PHPUnit
EXERCISES
1. What’s Unit Testing?
2. What are the main types of tests?
3. What are the benefits of writing the tests before the code?
4. What’s an assertion?
5. What’s a fixture?
6. What’s a data provider?
7. What’s the difference between a Stub and a Mock?
8. What’s the purpose of the phpunit.xml file?
Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/

More Related Content

PPT
Getting Started With Jenkins And Drupal
KEY
Continuous Integration & Drupal
PPTX
Drupal Continuous Integration with Jenkins - Deploy
PPTX
Best practices for Continuous Deployment with Drupal - DrupalCon Latin Améric...
ZIP
Drupal Deployment
PPTX
Drupal Continuous Integration with Jenkins - The Basics
PDF
DevOps: Cooking Drupal Deployment
PDF
IBM Drupal Users Group Discussion on Managing and Deploying Configuration
Getting Started With Jenkins And Drupal
Continuous Integration & Drupal
Drupal Continuous Integration with Jenkins - Deploy
Best practices for Continuous Deployment with Drupal - DrupalCon Latin Améric...
Drupal Deployment
Drupal Continuous Integration with Jenkins - The Basics
DevOps: Cooking Drupal Deployment
IBM Drupal Users Group Discussion on Managing and Deploying Configuration

What's hot (20)

PDF
Implementing CI CD UiPath Using Jenkins Plugin
KEY
Drupal Deployment
PDF
CI and CD Across the Enterprise with Jenkins (devops.com Nov 2014)
PDF
Journée DevOps : De l'intégration continue au déploiement continu avec Jenkins
PDF
Drupalcamp Simpletest
ODP
Continuous deployment-at-flipkart
PDF
At Your Service: Using Jenkins in Operations
PPTX
Jenkins and Chef: Infrastructure CI and Automated Deployment
ODP
7 Habits of Highly Effective Jenkins Users
PDF
Devops For Drupal
ODP
Jenkinsconf Presentation - Advance jenkins management with multiple projects.
PDF
Introduction to Infrastructure as Code & Automation / Introduction to Chef
PDF
PuppetConf 2016: Continuous Delivery and DevOps with Jenkins and Puppet Enter...
PDF
Drupal development, deployment, and automation with Puppet
PPTX
Automated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWS
PPTX
CI/CD Pipeline to Deploy and Maintain an OpenStack IaaS Cloud
PDF
Jenkins-CI
PPTX
Continuous Development with Jenkins - Stephen Connolly at PuppetCamp Dublin '12
KEY
Continuous Delivery Using Jenkins
PDF
Devops interview-questions-PDF
Implementing CI CD UiPath Using Jenkins Plugin
Drupal Deployment
CI and CD Across the Enterprise with Jenkins (devops.com Nov 2014)
Journée DevOps : De l'intégration continue au déploiement continu avec Jenkins
Drupalcamp Simpletest
Continuous deployment-at-flipkart
At Your Service: Using Jenkins in Operations
Jenkins and Chef: Infrastructure CI and Automated Deployment
7 Habits of Highly Effective Jenkins Users
Devops For Drupal
Jenkinsconf Presentation - Advance jenkins management with multiple projects.
Introduction to Infrastructure as Code & Automation / Introduction to Chef
PuppetConf 2016: Continuous Delivery and DevOps with Jenkins and Puppet Enter...
Drupal development, deployment, and automation with Puppet
Automated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWS
CI/CD Pipeline to Deploy and Maintain an OpenStack IaaS Cloud
Jenkins-CI
Continuous Development with Jenkins - Stephen Connolly at PuppetCamp Dublin '12
Continuous Delivery Using Jenkins
Devops interview-questions-PDF
Ad

Viewers also liked (6)

PDF
Drupal for Publishers: How to Build a Better Newsroom CMS
ZIP
First Steps in Drupal Code Driven Development
PDF
Best Practice Checklist for Building a Drupal Website
PDF
Drupal Module Development
PDF
Drupal 7
PDF
Softwarekonfiguration mit Drupal
Drupal for Publishers: How to Build a Better Newsroom CMS
First Steps in Drupal Code Driven Development
Best Practice Checklist for Building a Drupal Website
Drupal Module Development
Drupal 7
Softwarekonfiguration mit Drupal
Ad

Similar to Drupal Continuous Integration (European Drupal Days 2015) (20)

PDF
Сontinuous Integration - step to continuous deployment
PPT
Continuous integration
PDF
Let’s start Continuous Integration with jenkins
PDF
From 4 releases per year to 4 releases per day
PPTX
Drupal & Continous Integration - SF State Study Case
PDF
Continuous Integration @ MeetMagento Germany 2015
KEY
Continuous Integration In A PHP World
ODP
Continuous Integration and PHP
PPTX
Diy cont integration_dc_cebu
PPT
Continuous Integration: A Case Study
PPT
Continuous integration php
PPT
Php day 20 11 e xo continuousintegration php
PDF
Continuous Integration Is for Teams: Moving past buzzword driven development
PPTX
Continuous integration - ThangTD
PDF
Continuous Integration at T3CON08
PDF
Introduction to Continuous Integration
PDF
Continuous Integration from server to cloud
KEY
Hudson Continuous Integration for PHP
PPTX
Continuous Integration
PDF
10 tips for continuous integration
Сontinuous Integration - step to continuous deployment
Continuous integration
Let’s start Continuous Integration with jenkins
From 4 releases per year to 4 releases per day
Drupal & Continous Integration - SF State Study Case
Continuous Integration @ MeetMagento Germany 2015
Continuous Integration In A PHP World
Continuous Integration and PHP
Diy cont integration_dc_cebu
Continuous Integration: A Case Study
Continuous integration php
Php day 20 11 e xo continuousintegration php
Continuous Integration Is for Teams: Moving past buzzword driven development
Continuous integration - ThangTD
Continuous Integration at T3CON08
Introduction to Continuous Integration
Continuous Integration from server to cloud
Hudson Continuous Integration for PHP
Continuous Integration
10 tips for continuous integration

More from Eugenio Minardi (20)

PDF
Delphi and ExtJS (26 ottobre 2017)
PDF
ExtJS: La piattaforma vincente (tools)
PDF
ExtJS: La piattaforma vincente (multiple screens)
PDF
ExtJS: La piattaforma vincente (rich UI)
PDF
ExtJS: La piattaforma vincente (class system)
PDF
ExtJS: La piattaforma vincente
PDF
Distributed Team Management: 
Pitfall, Challenges and Advantages
PDF
A Practical Introduction to Symfony (European Drupal Days 2015)
PDF
UN World Food Programme Standards & Best Practises (European Drupal Days 2015)
PDF
Drupal theming - a practical approach (European Drupal Days 2015)
PDF
Optimizing MariaDB for Web Applications (European Drupal Days 2015)
PDF
PhpStorm for Drupal Development (European Drupal Days 2015)
PDF
Deploying an Open Source DAM in SAAS Mode (European Drupal Days 2015)
PDF
The multilingual Drupal 8 experience (European Drupal Days 2015)
PDF
Another Copernican Revolution: maintenance first, projects second (European D...
PDF
Drupal Security: How to survive Drupalgeddon and prepare for future (European...
PDF
The benefits of an elastic infrastructure on a Drupal e-commerce (European Dr...
PDF
Verifying Drupal modules with OWASP ASVS 2014 (European Drupal Days 2015)
PDF
Secure Drupal, from start to finish (European Drupal Days 2015)
PDF
Bridging the gap between business and technology - Behaviour Driven Developme...
Delphi and ExtJS (26 ottobre 2017)
ExtJS: La piattaforma vincente (tools)
ExtJS: La piattaforma vincente (multiple screens)
ExtJS: La piattaforma vincente (rich UI)
ExtJS: La piattaforma vincente (class system)
ExtJS: La piattaforma vincente
Distributed Team Management: 
Pitfall, Challenges and Advantages
A Practical Introduction to Symfony (European Drupal Days 2015)
UN World Food Programme Standards & Best Practises (European Drupal Days 2015)
Drupal theming - a practical approach (European Drupal Days 2015)
Optimizing MariaDB for Web Applications (European Drupal Days 2015)
PhpStorm for Drupal Development (European Drupal Days 2015)
Deploying an Open Source DAM in SAAS Mode (European Drupal Days 2015)
The multilingual Drupal 8 experience (European Drupal Days 2015)
Another Copernican Revolution: maintenance first, projects second (European D...
Drupal Security: How to survive Drupalgeddon and prepare for future (European...
The benefits of an elastic infrastructure on a Drupal e-commerce (European Dr...
Verifying Drupal modules with OWASP ASVS 2014 (European Drupal Days 2015)
Secure Drupal, from start to finish (European Drupal Days 2015)
Bridging the gap between business and technology - Behaviour Driven Developme...

Recently uploaded (20)

PPTX
Power Point - Lesson 3_2.pptx grad school presentation
PPT
FIRE PREVENTION AND CONTROL PLAN- LUS.FM.MQ.OM.UTM.PLN.00014.ppt
PDF
💰 𝐔𝐊𝐓𝐈 𝐊𝐄𝐌𝐄𝐍𝐀𝐍𝐆𝐀𝐍 𝐊𝐈𝐏𝐄𝐑𝟒𝐃 𝐇𝐀𝐑𝐈 𝐈𝐍𝐈 𝟐𝟎𝟐𝟓 💰
PPT
isotopes_sddsadsaadasdasdasdasdsa1213.ppt
PPTX
Internet___Basics___Styled_ presentation
PDF
Exploring VPS Hosting Trends for SMBs in 2025
PDF
An introduction to the IFRS (ISSB) Stndards.pdf
DOC
Rose毕业证学历认证,利物浦约翰摩尔斯大学毕业证国外本科毕业证
PPTX
E -tech empowerment technologies PowerPoint
PDF
Best Practices for Testing and Debugging Shopify Third-Party API Integrations...
PDF
Tenda Login Guide: Access Your Router in 5 Easy Steps
PDF
Slides PDF The World Game (s) Eco Economic Epochs.pdf
PPTX
SAP Ariba Sourcing PPT for learning material
PPTX
Introduction to Information and Communication Technology
PDF
The New Creative Director: How AI Tools for Social Media Content Creation Are...
PPTX
Job_Card_System_Styled_lorem_ipsum_.pptx
PDF
Unit-1 introduction to cyber security discuss about how to secure a system
PDF
Sims 4 Historia para lo sims 4 para jugar
PDF
SASE Traffic Flow - ZTNA Connector-1.pdf
PPTX
Funds Management Learning Material for Beg
Power Point - Lesson 3_2.pptx grad school presentation
FIRE PREVENTION AND CONTROL PLAN- LUS.FM.MQ.OM.UTM.PLN.00014.ppt
💰 𝐔𝐊𝐓𝐈 𝐊𝐄𝐌𝐄𝐍𝐀𝐍𝐆𝐀𝐍 𝐊𝐈𝐏𝐄𝐑𝟒𝐃 𝐇𝐀𝐑𝐈 𝐈𝐍𝐈 𝟐𝟎𝟐𝟓 💰
isotopes_sddsadsaadasdasdasdasdsa1213.ppt
Internet___Basics___Styled_ presentation
Exploring VPS Hosting Trends for SMBs in 2025
An introduction to the IFRS (ISSB) Stndards.pdf
Rose毕业证学历认证,利物浦约翰摩尔斯大学毕业证国外本科毕业证
E -tech empowerment technologies PowerPoint
Best Practices for Testing and Debugging Shopify Third-Party API Integrations...
Tenda Login Guide: Access Your Router in 5 Easy Steps
Slides PDF The World Game (s) Eco Economic Epochs.pdf
SAP Ariba Sourcing PPT for learning material
Introduction to Information and Communication Technology
The New Creative Director: How AI Tools for Social Media Content Creation Are...
Job_Card_System_Styled_lorem_ipsum_.pptx
Unit-1 introduction to cyber security discuss about how to secure a system
Sims 4 Historia para lo sims 4 para jugar
SASE Traffic Flow - ZTNA Connector-1.pdf
Funds Management Learning Material for Beg

Drupal Continuous Integration (European Drupal Days 2015)

  • 1. © Ibuildings 2014/2015 - All rights reserved #DrupalDaysEU DRUPAL CONTINUOUS INTEGRATION by Claudio Beatrice
  • 2. #DrupalDaysEU © Ibuildings 2014/2015 - All rights reserved Gold Sponsors
  • 3. #DrupalDaysEU © Ibuildings 2014/2015 - All rights reserved Media Sponsors Silver Sponsors
  • 4. DRUPAL CONTINUOUS INTEGRATION Introduction Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 5. Continuous Integration “Continuous Integration” is a software development practice where members of a team integrate work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including tests) to detect integration errors as quickly as possible. — Martin Fowler Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 6. Continuous Integration • automating repetitive tasks such as build, QA and deploy • fast and frequent integrations • fast feedback loop, catch errors asap • ability to deploy quickly and keep track of every released set of code ADVANTAGES Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014-2015http://tenwarp.com/
  • 7. Continuous Integration developer commits triggers config static analysis unit / functional tests runs runs runs report FAIL deploy smoke / functional tests report SUCCESS SUCCESS FAIL CI server revision control log logs logs logs BUILD PROCESS EXAMPLE Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 8. Continuous Integration TOOLS • SCM: git, svn, cvs, … • CI SERVER: jenkins • BUILD TOOL: phing • DRUPAL AUTOMATION: drush • DEPENDENCIES MANAGEMENT: drush make & composer • DRUPAL CDD TOOLS: features & migrate • TESTING: PHPUnit & Behat Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 9. Continuous Integration DRUPAL - I Drupal, as opposed to “classic” frameworks such as Symfony or Zend, does not make the setup and maintenance a Continuous Integration cycle easy due to the high coupling between data and configuration within its database. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 10. Continuous Integration DRUPAL - II To overcome this problem it’s necessary to adopt a strictly controlled, disciplined work methodology and to make use of a set of tools that allow us to transform configuration into code, to easily deal with content and to incrementally update database’s schema. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 11. Continuous Integration WORKFLOW - I A developer’s workflow consist not only in the plain old features implementation, but also in the correct, precise and complete export of all the element of configuration that are necessary to rebuild the functionalities on several environments. ! Key modules: Features and Strongarm. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 12. Continuous Integration WORKFLOW - II It’s often necessary to setup some default content that may vary during development and that might be expanded or updated once the site has been published. It’s therefore necessary to formalise in code all the data import procedures in order to better administrate them and to be able to replicate them at any given time. ! Key module: Migrate. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 13. Continuous Integration WORKFLOW - III As soon as a self-contained task such a bug-fix or a new feature implementation has been completed, it’s necessary to commit and push that unit of work to the central repository in order to integrate, test and deploy it automatically on the development environment. ! Key tools: git/svn. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 14. Continuous Integration WORKFLOW - IV ! Writing automated tests is necessary in order to guarantee the behaviour of a given implementation and to make sure that future changes won’t break the work already done, creating regressions and dealing potential damage to users and clients. ! Key tools: PHPUnit, Behat. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 15. Q & A Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 16. Continuous Integration EXERCISES 1. What are the main advantages of a Continuous Integration system? 2. When do Unit Tests are executed during the build? How about functional? Why are they executed at different times? 3. Why is it important to write tests? 4. Which benefits come from using a Source Control Management system? 5. Why is Drupal so hard -compared to other tools such as “classic” frameworks- to use in a Continuous Integration environment? 6. What tools allow Drupal to be successfully employed in a Continuous Integration development cycle and for what reasons? Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 17. DRUPAL CONTINUOUS INTEGRATION Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 18. Phing is a build system written in PHP that allows developers to describe the automated operations used during development, deployment and maintenance using plain XML files. Phing Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 19. • Simplicity: it’s based on well-known, declarative XML syntax and features a very rich documentation • Power: thanks to the big amount of available tasks, it is able to perform almost anything you could do using a command line interpreter • Extensibility: as it is written in PHP, it’s easy to extend and debug • Automation: the whole application configuration and deployment process can be handled by build files Phing BENEFITS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 20. • execute tests and log their results • perform filesystem operations • interact with source configuration management systems • generate documentation • transfer files from/to remote systems • fire drush commands • invoke shell commands Phing FEATURES Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 21. There are two available file types in Phing to configure the build: .properties and .xml files. The former contains a list of key/value variables, while the latter contains the “instructions” to execute. In a usual setup there are up to five different environments: loc, dev, stage, debug and prod. Each of these ones will be configured using a dedicated .properties file in order to make the build file environment-agnostic so to reuse most of its code. Phing MAIN FILES AND ENVIRONMENTS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 22. Phing PROPERTY FILE Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 23. Phing BUILD FILE - PROJECT TAG For every build file, there must be a single, unique, root “project" tag. Within that "target", "task" and “type" tags can be used. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 24. Phing BUILD FILE - TYPE TAG Besides default types such as numbers or strings, it is possible to define more complex, nested types. An interesting feature of Phing’s types is the ability to be referenced by id later. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 25. Phing BUILD FILE - TARGET TAG “Targets” are collections of components that are assigned a unique name within the project. It is possible to invoke other targets within a target and it’s also possible to set dependencies between different targets. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 26. Phing BUILD FILE - TASK TAG “Task”s are responsible for the execution of specific units of work and are usually configurable through attributes and nested tags. A few examples of what tasks do could be: files copying, archive compression or drush commands execution. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 27. Q & A Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 28. Phing EXERCISES 1. What are the main benefits of using a build system? 2. Why is Phing more convenient than a bunch of shell scripts? 3. What files does Phing use to work? What’s their use? 4. What are the fundamental elements of a build file? What’s their use? Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 29. DRUPAL CONTINUOUS INTEGRATION Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 30. Drush Drush is a command-line shell and scripting interface for Drupal, a veritable Swiss Army knife designed to make life easier for those who spend their working hours hacking away at the command prompt. — Drush Team Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 31. Drush • Drupal Power: ability to simply, swiftly and programmatically execute Drupal-specific tasks • Integration: working with other command-line tools is very easy and effective BENEFITS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 32. Drush • code and database backup, restore e sync • cron tasks execution • automatic installations • variables, cache and logs CRUD operations • modules and themes administration • SQL commands execution • users administration • dependency management • remote tasks execution FEATURES Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 33. Drush Drush make is a Drush extension that deals with modules, themes, libraries and patches management. It makes use of a manifest file to index and download the required versions of the packages used to build a Drupal website. MAKE - I Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 34. Drush MAKE - II Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 35. Drush Drush Make allows developers to specify a destination sub-folder using the “subdir” option. We’ll leverage that to establish the convention of having four sub-folders for modules: contrib, custom, devel and features. This division will help in keeping a cleaner division between the modules and will save us the burden of versioning “contrib” modules. MAKE - III Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 36. Drush Drush allows the developers to create a kind of labels called Site Aliases: these are nothing but configuration groups that fall under an arbitrary name and make the execution of some Drush commands easier. In our case those aliases match the environments defined for our project, and will look something like myproject.loc and myproject.dev. SITE ALIAS - I Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 37. Drush SITE ALIAS - II Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 38. Drush Drush looks for Aliases definitions in several locations: 1. in the path set in the “alias path” variable of the options array found in the drushrc.php file 2. in the path specified in the —alias-path command-line option 3. in one of the following folders: “/etc/drush”, “$HOME/.drush” or the current site’s “sites/all/drush” In order to keep Drush Aliases clean and separated, we’ll use our projects’ “sites/all/drush” as the go-to location to store their configuration so not to pollute the global configuration. SITE ALIAS - III Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 39. Drush Drush implements two extremely useful commands to keep files and databases in sync between hosts: rsync e sql-sync. • drush rsync @mysite.loc @mysite.dev • drush sql-sync @mysite.loc @mysite.dev SYNCHRONISATION Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 40. Q & A Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 41. Drush EXERCISES 1. What’s Drush? 2. Why is it convenient to use it? 3. What are its most important features? 4. What’s the use for a make file? 5. What’s a Drush Site Alias? What’s its use? Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 42. DRUPAL CONTINUOUS INTEGRATION Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 43. Jenkins Jenkins is a Java application useful to manage the execution of those repetitive tasks that are needed to build a software project. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 44. Jenkins • automation of deployments procedures • speed-up of repetitive tasks such as test runs or the creation of build artifacts to deploy on our projects’ target environments • logging and monitoring for a handful of code-related stats such as test results or code quality analysis • convenient web ui to configure all the needed jobs BENEFITS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 45. Jenkins FEATURES • integration with several source configuration management systems • tests results and code quality reports generation • integration with several build systems • jobs status notifications • configurable web ui • hundreds of plugins Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 48. Jenkins JOB CREATION - I Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 49. Jenkins JOB CREATION - II Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 50. Jenkins DASHBOARD Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 51. Q & A Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 52. Jenkins ESERCIZI 1. What’s Jenkins? 2. What are its tasks? 3. What’s a Job? Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 53. DRUPAL CONTINUOUS INTEGRATION Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 54. Features The Feature module is able to export parts of the configuration found in the database into “ordinary” Drupal modules. The ability to produce a set of files containing a specific functionality starting from the database makes them much easier to manage and deploy across several systems and installations. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 55. • Removes the need for manual, repetitive work that’s necessary to reproduce all of a website’s features and traits by encapsulating them in portable, standard Drupal modules; • Exports directly to Drupal modules or, in other words, text files: that means it’s easy to version them using a source control management system, which in turn makes easy to keep the history of changes and allows multiple people to work on the project concurrently; • Allows to transport functionalities across websites in an easy, reproducible and automatic fashion. Features BENEFITS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 56. Features Local Stage Dev Prod Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 57. Features CREATION - I Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 58. Features CREATION - II Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 59. Features ADMINISTRATION Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 60. Features OPERATIONS - ENABLING A FEATURE Enabling a Features-exported module executes the same operations a “normal” module executes, such as: • Schema installation; • System Register and Cache update; • Execution of hook_install(); • Execution of hook_enable(). Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 61. Features OPERATIONS - UPDATE & REVERT Features Update: updating a feature consists in the act of saving some of the information found in the database into a specific Feature Module. Features Revert: reverting a feature consists in the act of writing the information found in a Feature Module into the database. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 62. Features features-components (fc) List features components. features-diff (fd) Show the difference between the default and overridden state of a feature. features-export (fe) Export a feature from your site into a module. features-list (fl) List all the available features for your site. features-revert (fr) Revert a feature module on your site. features-revert-all (fra) Revert all enabled feature module on your site. features-update (fu) Update a feature module on your site. features-update-all (fua) Update all feature modules on your site. DRUSH COMMANDS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 63. Features • Thanks to the power and flexibility granted by the Entity API it’s possible to easily create exportable Entities that are automatically picked up and handled by Features; • A lot of contrib modules are already using such possibility in order to provide exportable configuration. Some examples could be the Rules, Views or Context modules. BEYOND THE DEFAULT OPTIONS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 64. Features • Create and maintain “vertical” features such as blog, forum and front page. Avoid creating Feature Modules that group “exportables” by type, such as “all site’s views” or “all site’s rules”; • Do always use English and specific namespaces when naming components (field_blog_topics instead of field_argomenti); • Do not use abbreviations unless it’s strictly ncssry. • Keep a standard format when naming fields: try not to mix casing, dashing, underscores and so on; • Define conventions. The Kit Project, although abandoned, is a good source of inspiration; • Strive to keep things clean and in order: try to always use Features for importing and exporting all the developed functionalities. BEST PRACTICES Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 65. Q & A Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 66. Features EXERCISES 1. What are the main goals of the Features module? 2. What components can it act upon? 3. What does feature-revert do? 4. What benefits does Features bring? 5. What’s Strongarm used for? 6. Can you name a few modules that allow their components to be exported by Features? Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 67. DRUPAL CONTINUOUS INTEGRATION Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 68. Migrate The Migrate module makes a powerful and flexible object oriented framework for importing contents into Drupal available to developers: it offers out-of- the-box support for a handful of sources, such as databases and file formats, and destinations, such as nodes and taxonomy terms. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 69. Migrate At macro-level, there’s four interesting areas to consider when working with Migrate: • An API for defining and managing the migration process; • A set of Drush commands for managing the migrations, such as import, rollback and status; • An Admin UI featuring some of the features exposed by Drush; • Some example modules that demonstrate a handful of techniques used to define custom migrations. FEATURES Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 71. Migrate SOURCE • Inherits from the abstract class MigrateSource • Provides an interface to the data set • Iterates on the source’s rows • Provides a list of fields Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 72. Migrate DESTINATION • Inherits from the abstract class MigrateDestination (and usually from the more specific MigrateDestinationEntity) • Responsible for persisting data in a specific Drupal’s content type such as user, node, comment, file. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 73. Migrate MAPPING TABLE • Inherits from MigrateSQLMap, which in turn inherits from the abstract class MigrateMap • Describes the key formats for the source and the destination • Stores all the information that’s used to import and rollback each and every single record Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 74. Migrate FIELDS MAPPING • Allows to link each destination field with a source field • Allows to transform data during the migration to fix or adapt to the destination requirements. • Allows to exclude some fields from the migration Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 75. Migrate Handlers are classes that allow to execute additional operations during the import process. They are usually used to support migrations where some specific, non-standard data is connected to Core Entities: most of the time this will be related to custom field types. HANDLER Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 76. Migrate There’s two kinds of Handler: Destination and Field. The former is used to manipulate the Destination Entity around saving (pre- and post- save) and to declare additional fields exposed by the Destination Entity. The latter works much like the Destination Handler, except its methods are invoked for each Field (as opposed to each Entity). HANDLER Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 77. Migrate Sometimes there are pieces of content that hold circular references to each other: for instance a tutorial post “part 1” could contain a reference to the “part 2” and vice-versa. STUB NODE nid 1 title tutorial part I body … series_nids 2,3,4 NODE nid 2 title tutorial part II body … series_nids 1,3,4 Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 78. Migrate During the import process the first node won’t be able to correctly link to the second as the (new) id is still unknown: Stubs solve this problem by creating and linking a temporary, empty node that only holds an id. As soon as the import process gets to the stubbed node, it will populate it with all the due information. STUB Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 79. Migrate Most of the time migrations are defined as static as that source and the destination match one-to-one: one row, one migration. However, sometimes it’s necessary to “join” multiple sources having the same structure into a single destination: this kind of migration is defined as dynamic. An example could be a migration where you need to import users from several Drupal sites into a new one. DYNAMIC MIGRATIONS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 80. Migrate When using this functionality, Drupal will take care of instantiating each migration multiple times. Besides that, a dynamic migration differs from a static migration in the following ways: • its constructors accepts an array of arguments, where each of its keys identifies an instance of the migration class; • it overrides the generateMachineName() method in order to generate a distinct machine name for each instance, so to be able to identify them. DYNAMIC MIGRATIONS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 81. Migrate migrate-analyze (maz) Analyze the source fields for a migration. migrate-audit (ma) View information on problems in a migration. migrate-auto-register (mar) Register any newly-defined migration classes. migrate-deregister Remove all tracking of a migration. migrate-fields-destination (mfd) List the fields available for mapping in a destination. migrate-fields-source (mfs) List the fields available for mapping from a source. migrate-import (mi) Perform one or more migration processes. migrate-mappings View information on all field mappings in a migration. migrate-messages (mmsg) View any messages associated with a migration. migrate-reset-status (mrs) Reset a active migration's status to idle. migrate-rollback (mr) Roll back the destination objects from a given migration. migrate-status (ms) List all migrations with current status. migrate-stop (mst) Stop an active migration operation. migrate-wipe (mw) Delete all nodes from specified content types. DRUSH COMMANDS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 83. Migrate • Migrate D2D - set of utilities for migrating content from other versions of Drupal (5, 6, 7) to Drupal 7; • Migrate Extras - integrates several contrib modules such as Rules or Pathauto; • Commerce Migrate - Integrates Drupal Commerce’s Entities; • Wordpress/Typo3/PHPBB Migrate - set of utilities to aid the import of content from other well-known platforms. BEYOND THE DEFAULT OPTIONS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 84. Q & A Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 85. Migrate EXERCISES 1. What’s the main goal of the Migrate module? 2. What could be the sources for a migration? 3. What’s the use for the mapping table? 4. What does migrate-rollback do? 5. When is appropriate to use Migrate? 6. What’s a dynamic migration? How is it used? 7. What’s a stub? Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 86. DRUPAL CONTINUOUS INTEGRATION Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 87. Behat Behat is a testing framework written in PHP with the purpose of aiding the writing of Acceptance Tests using Behaviour Driven Development practices. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 88. Behat IT’S NOT A FUNCTIONAL TESTING FRAMEWORK It’s a tool to verify that a given system satisfies the business needs of the client. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 89. Behat Tests are written using a semi-formal language called Gherkin. In a nutshell, Behat maps Gherkin sentences -usually written in English, but it could be any language- to a class method using to regular expressions. GHERKIN Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 90. Behat GHERKIN FEATURE Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 91. Behat StoryBDD Behaviour Driven Development based on User Stories -aka StoryBDD- is a practice that helps into bringing a development team’s understanding of business requirements to the same level of the client’s. It does so by setting a formal process for asking questions about requirements and then expressing the outcome in a specific story-format. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 92. Behat For each new feature, a developer should answer the following business questions: • Who will use it? • What’s its purpose? • How is that achieved? Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/ StoryBDD
  • 93. Behat “I want users to access the site through a dedicated form” • Who will use it? a user an anonymous visitor • What’s the purpose? it’s not stated • How is that achieved? by using the login form Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/ StoryBDD
  • 94. “I want an anonymous visitor to be able to access the site using the login form in order to access her dashboard containing her aggregated data” ! In order to see aggregated data on my dashboard as a site visitor I need to be able to login on the site Behat benefit role condition Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/ StoryBDD
  • 95. Once a feature is defined, you should start writing all the scenarios that describe it, using the Context- Action-Outcome pattern: ! Given I am in on the homepage When I login Then I should see “Welcome Back!” And I should be on my dashboard Behat SCENARIOS context action outcome Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 96. Behat Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 97. Behat BEST PRACTICES And I should see “message” inside “.notifications > p:first-child” ! It’s recommended to avoid technical details in your scenarios as they increase the coupling between tests and code and make the stories harder for non-technical readers -such as the client- to understand. ! And I should see “message” inside notification area Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 98. Behat MINK Each test gets executed within a context where the methods invoked by the expressions used in the scenarios can access to a browser-abstraction API called Mink. There are several drivers available for Mink such as Goutte, a headless browser written in PHP that’s able to parse HTML and XML and Selenium, a full-fledged framework able to programmatically control real browsers. Oh and there’s a Drush Driver, too! :) Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 99. Behat MINK CONTEXT Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 100. Behat MINK BEST PRACTICES The Page object pattern is a way of keeping context files clean by separating UI knowledge from the actions and assertions. In other words, page objects hide the UI and expose clean services that can be used in the context classes: such abstraction makes context classes much more clear and helps avoiding duplication and improving maintainability. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 101. Behat MINK DRUSH DRIVER Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 102. Behat EXECUTION EXAMPLE Gherkin Feature Behat Mink Goutte Selenium Firefox Test Results Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 103. Q & A Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 104. Behat EXERCISES 1. What’s Behat and what’s its main goal? 2. What’s StoryBDD? 3. What’s a scenario? 4. What languages can be used to write the tests? 5. What questions should be answered by a feature description? 6. What are Mink’s main tasks? 7. What browsers can be used run the tests? What are the main differences between them? Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 105. DRUPAL CONTINUOUS INTEGRATION Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 106. PHPUnit PHPUnit is a unit testing framework, it’s part of xUnit family and it has become the de-facto PHP standard over the years. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 107. PHPUnit In software engineering, unit testing is a method by which individual units of code are tested to determine if they are fit for use. A unit is defined as the smallest part of an application that can be tested in isolation: most of the time that unit is a class or a method. UNIT TESTING Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 108. PHPUnit A SUT -an acronym for System Under Test- is defined as that subset of the software being developed that’s exercised by a test. It can be a specific method, and object or even a set of objects. DEFINITIONS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 109. PHPUnit Writing automated tests (not necessarily unit tests only) sets some goals: • Improving the quality of the product and its code; • Increasing developers’s understanding of the SUT; • Decreasing the damage caused by unexpected bugs; • Simplifying the creation, maintenance and execution of the tests; • Documenting the behaviours of the applications’ components. GOALS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 110. PHPUnit TYPES OF TESTS Unit Integration Functional Functional: they operate on the system from the outside, often by simulating/employing a browser. Integration: they check how multiple components of the system cooperate when exercised together. Unit: they make sure that the single units of code behave as expects when exercised in isolation. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 111. PHPUnit Should I test before or after development? Traditionally tests -being them unitary or functional- have always been executed once the implementation was completed. Such approach has been upturned by the rise of agile methodologies, where the emphasis is put on the ability to respond to change quickly rather than sticking to a plan. Writing tests first makes code inherently testable and therefore safer to modify and usually produces less verbose and over- engineered implementations. METHODOLOGIES Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 112. PHPUnit Should I test outside-in or inside-out? As soon as development begins, a very important decision has to be made: should it start from the big picture and “zoom in” or from one or some of the details and “zoom out” in order to compose the macro-functionality at the end? METHODOLOGIES Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 113. PHPUnit Should I test outside-in or inside-out? By starting development(and testing) from the outside, the developer forces himself to think as a client first, a method that helps in making requirements and (temporarily missing) dependencies more explicit. Developing “from the inside” on the other hand frees the developer from the problem of not existing dependencies at the cost of guessing what’s going to be the right set of things to implement before reaching the requested level of functionality. METHODOLOGIES Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 114. PHPUnit Testing status or behaviour? There are two main ways to verify the correctness of a SUT: • Testing the state: it’s often sufficient to verify that given an initial state, once the system is exercised the ending state then matches the expected one. • Testing the behaviour: it’s sometimes necessary to look inside the black box of the SUT in order to check how the system works during the execution of the tests to make sure all the behavioural expectations were satisfied. METHODOLOGIES Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 115. PHPUnit TOOLS PHPUnit provides a number of tools and helpers to aid testing, such as: • Assertions • Fixtures • Database testing helpers • Stub and mock objects • Code coverage analysis tools • Several logging formats • Integration with several functional testing tools such as Selenium Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 116. PHPUnit EXAMPLE Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 117. PHPUnit PHPUnit features about 80 assertions (of which 40 are “straight” and 40 are “negated”). It’s up to the developer to pick the best one for the test in order to have the most expressive tests and the best error messages possible. ASSERTIONS Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 118. PHPUnit FIXTURES When writing tests, one of the most tedious and time consuming parts is the preparation of the environment status before the actual execution happens: such status is known as the fixture of the test . PHPUnit features quite a big set of methods used to aid in the creation and sharing of these states, such as: setUpBeforeClass(), setUp(), assertPreConditions(), assertPostConditions(), tearDown(), tearDownAfterClass() and onNotSuccessfulTest(Exception $e). Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 119. PHPUnit DATA PROVIDERS It can be useful to be able to run multiple iterations of the same test on a larger-than-one data set. Data Providers allows for a convenient way to do just that. Such providers are nothing but a method that returns an array containing all the combinations of arguments a tests will be called with. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 120. PHPUnit TEST DOUBLES It could be helpful -or necessary- to replace a SUT’s dependencies with so-called “test doubles”, a kind of objects that looks like the dependency but it’s created with the sole purpose of satisfying testing needs. Such a replacement allows to isolate the SUT and manipulate its inputs as needed, making the test more effective and focused on the unit alone. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 121. PHPUnit TEST DOUBLES There are various kinds of Test Doubles, each serving a specific purpose. Meszaros and Fowler suggested the following five definitions: • Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists; • Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an InMemoryTestDatabase is a good example); Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 122. PHPUnit TEST DOUBLES • Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test; • Spies are stubs that also record some information based on how they were called. One form of this might be an email service that records how many messages it was sent; • Mocks are pre-programmed with expectations which form a specification of the calls they are expected to receive. They can throw an exception if they receive a call they don't expect and are checked during verification to ensure they got all the calls they were expecting. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 123. PHPUnit TEST DOUBLES Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 124. PHPUnit OPTIONS PHPUnit features a long list of options to configure its behaviour that’s accessible both from command line and from a configuration file. The former is a convenient way to quickly get started and try thing out, but on the long run the phpunit.xml file is probably a more convenient choice to configure the tool. It’s a good practice to commit a phpunit.xml.dist file and (git) ignore the phpunit.xml one: by doing such thing the other developers will be able to create a phpunit.xml file tailored to their own needs. PHPUnit will automatically use phpunit.xml if present or phpunit.xml.dist if not. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 125. PHPUnit OPTIONS BEST PRACTICES It’s a good practice to commit a phpunit.xml.dist file and (git) ignore the phpunit.xml one: by doing such thing the other developers will be able to create a phpunit.xml file tailored to their own needs, leaving the phpunit.xml.dist targeted at automatic builds, fore example. PHPUnit is smart enough to automatically use phpunit.xml if present or phpunit.xml.dist if not. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 127. PHPUnit BEYOND THE DEFAULT OPTIONS There are more tools provided by the community than what’s implemented by PHPUnit alone. Here’s a short list of some relevant ones: • Faker - a library that generates fake-yet-plausible data for a handful of typical use cases such as names, addresses, numbers, dates, images, etc… • Mockery - a powerful mocks creation library; • Prophecy - a new, strongly-opinionated mocks creation library; • PHP-VCR - a library to record and replay HTTP calls in tests; • TuTu - a simple and flexible HTTP mocking tool. Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 128. Q & A Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/
  • 129. PHPUnit EXERCISES 1. What’s Unit Testing? 2. What are the main types of tests? 3. What are the benefits of writing the tests before the code? 4. What’s an assertion? 5. What’s a fixture? 6. What’s a data provider? 7. What’s the difference between a Stub and a Mock? 8. What’s the purpose of the phpunit.xml file? Drupal Continuous Integration - By Claudio Beatrice - Copyright 2014http://tenwarp.com/