Posts in this category
- Automating Deployments: A New Year and a Plan
- Automating Deployments: Why bother?
- Automating Deployments: Simplistic Deployment with Git and Bash
- Automating Deployments: Building Debian Packages
- Automating Deployments: Debian Packaging for an Example Project
- Automating Deployments: Distributing Debian Packages with Aptly
- Automating Deployments: Installing Packages
- Automating Deployments: 3+ Environments
- Architecture of a Deployment System
- Introducing Go Continuous Delivery
- Technology for automating deployments: the agony of choice
- Automating Deployments: New Website, Community
- Continuous Delivery for Libraries?
- Managing State in a Continuous Delivery Pipeline
- Automating Deployments: Building in the Pipeline
- Automating Deployments: Version Recycling Considered Harmful
- Automating Deployments: Stage 2: Uploading
- Automating Deployments: Installation in the Pipeline
- Automating Deployments: Pipeline Templates in GoCD
- Automatically Deploying Specific Versions
- Story Time: Rollbacks Saved the Day
- Automated Deployments: Unit Testing
- Automating Deployments: Smoke Testing and Rolling Upgrades
- Automating Deployments and Configuration Management
- Ansible: A Primer
- Continuous Delivery and Security
- Continuous Delivery on your Laptop
- Moritz on Continuous Discussions (#c9d9)
- Git Flow vs. Continuous Delivery
Wed, 06 Jan 2016
Automating Deployments: Building Debian Packages
I have argued before that it is a good idea to build packages from software you want to automatically deploy. The package manager gives you dependency management as well as the option to execute code at defined points in the installation process, which is very handy for restarting services after installation, creating necessary OS-level users and so on.
Which package format to use?
There are many possible package formats and managers for them out there. Many ecosystems and programming languages come with their own, for example Perl uses CPAN.pm or cpanminus to install Perl modules, the NodeJS community uses npm, ruby has the gem installer, python pip and easyinstall, and so on.
One of the disadvantages is that they only work well for one language. If you or your company uses applications uses software in multiple programming languages, and you chose to use the language-specific packaging formats and tools for each, you burden yourself and the operators with having to know (and being aware of) these technologies.
Operating teams are usually familiar with the operating system's package manager, so using that seems like an obvious choice. Especially if the same operating system family is used throughout the whole organization. In specialized environments, other solutions might be preferable.
What's in a Debian package, and how do I build one?
.deb file is an
ar archive with meta data about
the archive format version, meta data for the package (name, version,
installation scripts) and the files that are to be installed.
While it is possible to build such a package directly, the easier and much
more common route is to use the tooling provided by the devscripts package. These tools
expect the existence of a
debian/ directory with various files in
debian/control contains information such as the package
name, dependencies, maintainer and description.
debian/rules is a
makefile that controls the build process of the debian package.
debian/changelog contains a human-readable summary of changes to
the package. The top-most changelog entry determines the resulting version of
You can use
dh_make from the
dh-make package to
generate a skeleton of files for the
debian/ directory, which you
can then edit to your liking. This will ask your for the architecture of the
package. You can use a specific one like
amd64, or the word
any for packages that can be build on any architecture. If
resulting package is architecture independent (as as is the case for many
scripting languages), using
all as the architecture is
Build process of a Debian package
If you use
dh_make to create a skeleton,
debian/rules mostly consists of a catch-all rule that calls
dh $@. This is tool that tries to do the right thing for each
build step automatically, and usually it succeeds. If there is a
Makefile in your top-level directory, it will call the
install make targets for you. If your build system installs into
DESTDIR prefix (which is set to
debian/your-package-name), it should pretty much work out of the
If you want to copy additional files into the Debian package, listing the
file names, one on each line, in
debian/install, that is done
automatically for you.
If you have already packaged your code for distribution through language-specific tools, such as CPAN (Perl) or pip (Python), there are shortcuts to creating Debian Packages.
dh-make-perl (installable via the package of the same
name) can automatically create a
debian directory based on the
perl-specific packaging. Calling
dh-make-perl . inside the root
directory of your perl source tree is often enough to create a functional
Debian package. It sticks to the naming convention that a Perl package
Awesome::Module becomes libawesome-module-perl in Debian land.
py2dsc from the
python-stdeb package generates a
debian/ directory from an existing python tarball.
Another approach is to use dh-virtualenv. This copies all of the python dependencies into a virtualenv, so the resulting packages only depends on the system python and possible C libraries that the python packages use; all python-level dependencies are baked in. This tends to produce bigger packages with fewer dependencies, and allows you to run several python programs on a single server, even if they depend on different versions of the same python library.
dh-virtualenv has an unfortunate choice of default installation prefix that
clashes with some assumptions that Debian's python packages make. You can
override that choice in
#!/usr/bin/make -f export DH_VIRTUALENV_INSTALL_ROOT=/usr/share/yourcompany %: dh $@ --with python-virtualenv --with systemd
It also assumes Pyhton 2 by default. For a Python 3 based project, add these lines:
override_dh_virtualenv: dh_virtualenv --python=/usr/bin/python3
(As always with Makefiles, be sure to indent with hard tabulator characters, not with spaces).
I'm writing a book on automating deployments. If this topic interests you, please sign up for the Automating Deployments newsletter. It will keep you informed about automating and continuous deployments. It also helps me to gauge interest in this project, and your feedback can shape the course it takes.