Skip to content

Commit 90cd9f0

Browse files
authored
Some modernization / updates / cleanup (#4)
- Updated to use version 3.0 of docker rotate - Changed install of python deps to happen in a single "pip" step using a requirements file; otherwise separate pip steps might result in unwanted versions of those dependencies. - Added support for logging into docker registries, based on config variable. - Changed pre-pull of packages to use a config variable, not a hardcoded list. - Updated to do log rotation by size, not time; use built-in Docker rotation for Docker v1.8+ - Removed step that installed inotify-tools. - Detect docker version instead of relying on "docker_version" variable. TIL-2663
1 parent db77ba9 commit 90cd9f0

File tree

9 files changed

+174
-87
lines changed

9 files changed

+174
-87
lines changed

CHANGES.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,16 @@
11
# Release Change Log
2+
Version 3.0:
3+
- use version 3.0+ of docker rotate
4+
- install docker-py, docker-compose, and dockerrotate in a single pip command to avoid
5+
them stepping on each other version-wise
6+
- log into Docker registries according to config variable.
7+
- pre-pull packages according to config variable, not hardcoded list.
8+
- do log rotation by size, not by time; use Docker's build-in rotation for Docker version 1.8+
9+
- no longer install inotify-tools
10+
211
Version 2.3:
312
- Update 'clear-zombie' script
4-
13+
514
Version 2.2:
615
- restart docker with new config before prefetching images
716

README.md

Lines changed: 50 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,62 @@
1-
docker base
2-
===========
1+
# docker base
32

43
This role installs baseline needs for machines running Docker, beyond the Docker Engine itself,
54
including:
65

76
- Requirements for running Ansible's `docker` module
8-
- Requirements for running Docker containers via `upstart` (optional)
97
- Utilities for garbage collecting unused Docker resources
108
- Customized configurations of the Docker Engine
11-
- Installs docker-compose
9+
- Installs docker-compose (optional)
1210
- Pulls some common images that we anticipate using in many circumstances
1311

14-
Role Variables
15-
--------------
12+
## Role Variables
1613

1714
This role expects the following variables:
1815

19-
- `docker_log_rotate_interval` controls the frequency of Docker log rotation
20-
- `docker_log_rotate_count` controls the number of Docker logs to retain
21-
- `docker_uses_upstart` defines whether `upstart` support will be installed
22-
- `docker_py_version` defines the version of the Docker Python library being used
23-
- `docker_compose_version` defines the version of docker-compose to install (defaults to newest)
24-
25-
Note that certain versions of the Docker Engine and Docker Python library are incompatible.
26-
27-
28-
Dependencies
29-
------------
30-
31-
Depends on `docker` and `python`
16+
- `docker_py_version` defines the version of the Docker Python library to install, may be
17+
"latest", default 1.9.0
18+
- `docker_rotate_version` defines the version of the Docker Rotate library to install, default 3.0
19+
- `docker_compose_version` defines the version of Docker Compose to install, may be "latest",
20+
default 1.8.0. To skip installation of docker_compose, specify `null` or `""`
21+
- `docker_rotate_images_arguments`: arguments to pass to "docker-rotate images" when cleanup cron
22+
job runs. If blank, job will not be run. Default is "--keep 3 --name ~busybox --tag ~latest".
23+
For documentation, see the
24+
[docker rotate github page](https://github.com/locationlabs/docker-rotate)
25+
- `docker_rotate_untagged_images`: if True, then untagged images will be periodically cleaned up.
26+
Default False
27+
- `docker_rotate_exited_containers`: if non-empty, exited containers at least this old will be
28+
removed when daily cleanup runs. Default 1h.
29+
`docker_rotate_created_containers`: if non-empty, created containers - that is, containers that
30+
have been created but never started - at least this old will be removed when daily cleanup runs.
31+
Default is empty.
32+
- `docker_log_rotate_max_size`: Docker's JSON logs for containers will be rotated when they reach
33+
this size. Default 10M.
34+
- `docker_log_rotate_count`: log files will be rotated this number of times before removal.
35+
- `docker_images_to_pull`: list of Docker images; images with these names will be "docker pull"ed
36+
when the role is run. Default is empty.
37+
- `docker_registries_to_login`: list of maps, each map containing login credentials for a Docker
38+
registry. Expected keys are `username`, `password`, `email`, and `url`. The role will attempt to
39+
log into each registry described in the list. Default is empty.
40+
41+
Example for `docker_registries_to_login`:
42+
43+
docker_registries_to_login:
44+
- username: foo
45+
password: foopwd
46+
47+
url: https://registry.foo.com
48+
49+
### Docker version notes
50+
Note that certain versions of the Docker Engine and Docker Python library are incompatible. Note
51+
also that the Ansible "docker" module has compatibility issues.
52+
53+
Specifically:
54+
- `docker-py` version 1.1.0 is the highest version that works with `docker` 1.5.0
55+
- `docker-py` version 1.2.3 is the highest version that works with the `docker` module in
56+
Ansible < 2.0.
57+
58+
## Dependencies
59+
In order to run this role:
60+
- python and pip must be installed.
61+
- docker engine must be installed. For a role that installs docker engine, see
62+
[here](https://github.com/locationlabs/ansible-role_docker)

defaults/main.yml

Lines changed: 38 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,6 @@
11
---
22

3-
# Should upstart be used to manage Docker containers?
4-
#
5-
# If upstart is not used, Docker containers should be run with
6-
# the "always" restart policy.
7-
#
8-
# If upstart is used, some extra dependencies will be required
9-
# (e.g. inotify-tools)
10-
11-
docker_uses_upstart: yes
12-
13-
14-
# Version of docker-py library to use.
3+
# Version of docker-py library to use. "latest" to install the latest version.
154
#
165
# The 1.1.0 version is the most recent version that works with Docker 1.5.0;
176
# it has also been shown to work with newer Docker versions (including 1.8.1).
@@ -21,31 +10,53 @@ docker_uses_upstart: yes
2110
#
2211
# Versions after 1.2.3 stopped working with the current Ansible docker module
2312
# (as of Ansible 1.9.2).
13+
docker_py_version: "1.9.0"
2414

25-
docker_py_version: 1.2.3
26-
15+
# Version of docker-compose to use.
16+
# 'latest' to install the newest version, null or empty string to install nothing.
17+
docker_compose_version: "1.8.0"
2718

28-
# Version of docker-rotate to use.
19+
# Version of docker-rotate to use. Versions lower than 3.0 are not supported due
20+
# to API changes.
2921
#
3022
# https://github.com/locationlabs/docker-rotate
31-
docker_rotate_version: 2.0.1
23+
docker_rotate_version: "3.0"
3224

25+
# These properties govern Docker Rotate configuration. https://github.com/locationlabs/docker-rotate
3326

34-
# Python regex of image names to remove. Use a '~' prefix for negative match.
35-
#
36-
# https://github.com/locationlabs/docker-rotate
37-
docker_rotate_containers_images_regex: ~busybox
27+
# Arguments to pass to "docker-rotate images" - pass False or empty string to not call command at
28+
# all .
29+
docker_rotate_images_arguments: "--keep 3 --name ~busybox --tag ~latest"
3830

39-
# Log rotate interval. Log files will be rotated after this interval.
31+
# Should docker-rotate remove untagged images?
32+
docker_rotate_untagged_images: True
4033

41-
docker_log_rotate_interval: "daily"
34+
# How long should docker-rotate retain exited containers? Specify a duration to keep containers
35+
# that long, "0" to remove them immediately, or an empty string to skip cleanup.
36+
docker_rotate_exited_containers: "1h"
4237

38+
# How long should docker-rotate retain "created" containers? Specify a duration to keep containers
39+
# that long, "0" to remove them immediately, or an empty string to skip cleanup.
40+
# Note that old-style "data" containers are in the "created" state.
41+
docker_rotate_created_containers: ""
4342

44-
# Log files are rotated <log_rotate_count> times before being removed.
45-
# If count is 0, old versions are removed rather then rotated.
43+
# In Docker <1.8, log rotation is implemented using logrotate; in Docker 1.8+, default Docker
44+
# settings are used.
45+
46+
# log rotation max size. Log files will rotate when they reach this size.
47+
docker_log_rotate_max_size: 10M
4648

49+
# Log files are rotated <log_rotate_count> times before being removed.
50+
# If count is 0, old versions are removed rather then rotated.
4751
docker_log_rotate_count: 3
4852

49-
# Version of docker-compose to use.
50-
# 'latest' to install the newest version
51-
docker_compose_version: latest
53+
# List of docker images to pull (at latest version) when role is run.
54+
docker_images_to_pull: []
55+
56+
# list of Docker registries to log in to; this is a list of dicts, eg:
57+
# docker_registries_to_login:
58+
# - username: foo
59+
# password: foopwd
60+
61+
# url: https://registry.foo.com
62+
docker_registries_to_login: []

tasks/main.yml

Lines changed: 46 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,70 @@
11
---
22

3-
- name: install inotify-tools
4-
apt: pkg=inotify-tools force=yes
5-
when: docker_uses_upstart
3+
# The role knows how to write docker_rotate commands for the version 3.0 API, not for earlier
4+
# versions.
5+
- name: check minimum docker_rotate version
6+
fail: msg="specified docker_rotate version is {{ docker_rotate_version }}, must be at least 3.0"
7+
when: "{{ docker_rotate_version | version_compare('3.0', '<') }}"
68

7-
- name: install docker-py
8-
pip: name=docker-py version="{{ docker_py_version }}"
9+
# Note escaping required here - we need to pass "{{ .Server.Version }}" to Docker without it
10+
# being escaped by Ansible. However, normal ways of escaping it will result in the command
11+
# containing that string, and when we register the result the "cmd" field on the result will also
12+
# contain it. So next time any kind of templating command is run, Ansible will try to resolve
13+
# template expressions in all of its variables, including that one, and will show an error.
14+
# Hence this workaround with shell.
15+
# In Ansible 2.0, they added the "!unsafe" keyword, which may provide a cleaner way to resolve
16+
# this.
17+
- name: Verify that docker is installed and determine installed version
18+
shell: 'B="{" && docker version -f "$B$B .Server.Version }}"'
19+
changed_when: False
20+
always_run: True
21+
register: docker_version_result
922

10-
- name: install docker-rotate
11-
pip: name=dockerrotate version="{{ docker_rotate_version}}"
23+
- name: store installed docker version
24+
set_fact:
25+
installed_docker_version: "{{ docker_version_result.stdout }}"
26+
27+
# install Python packages with pip. These are all included in the same requirements file, so that
28+
# we can let Pip sort out any inconsistencies between required versions.
29+
- name: manage python requirements file
30+
template: src=requirements.txt.j2 dest=/etc/docker-python.requirements.txt
31+
32+
# note state=latest here; if you want a fixed version, specify it.
33+
- name: ensure python reqirements are installed
34+
pip: state=latest requirements=/etc/docker-python.requirements.txt
1235

1336
- name: install docker-rotate cron script
14-
template: src=docker-rotate dest=/etc/cron.daily/ mode=755
37+
template: src=docker-rotate.j2 dest=/etc/cron.daily/docker-rotate mode=755
1538

1639
- name: push logrotate config file for docker container json logs
1740
template: src=logrotate.conf.j2 dest=/etc/logrotate.d/docker
41+
when: "{{ installed_docker_version | version_compare('1.8', '<') }}"
42+
43+
- name: make sure docker logrotate config is removed
44+
file: state=absent dest=/etc/logrotate.d/docker
45+
when: "{{ installed_docker_version | version_compare('1.8', '>=') }}"
1846

1947
- name: install docker zombie volumes cleanup cron script
2048
copy: src=clear-zombie-docker-data-dirs dest=/etc/cron.daily/ mode=755
2149

2250
- name: push docker configuration
2351
template: src=docker.j2 dest=/etc/default/docker
52+
register: docker_config_result
2453
notify: restart docker
2554

26-
- name: install docker-compose
27-
pip:
28-
name: docker-compose
29-
state: "{{ (docker_compose_version == 'latest') | ternary('latest', 'present') }}"
30-
version: "{{ (docker_compose_version == 'latest') | ternary(omit, docker_compose_version) }}"
31-
when: docker_compose_version is defined
32-
3355
# make sure docker restarts to get the newest configuration
3456
- meta: flush_handlers
3557

58+
- name: gather docker facts
59+
setup:
60+
when: "{{ ansible_docker0 is not defined or docker_config_result | changed }}"
61+
3662
- name: pull useful images for future use
3763
command: docker pull {{ item }}
38-
register: docker_result
64+
register: docker_pull_result
3965
changed_when: '"Downloaded newer image for" in docker_result.stdout'
40-
with_items:
41-
- busybox
42-
- ubuntu-debootstrap:14.04
66+
with_items: "{{ docker_images_to_pull }}"
4367

68+
- name: login to docker registry
69+
command: "docker login -u {{ item.username }} -p {{ item.password }} -e {{ item.email }} {{ item.url }}"
70+
with_items: "{{ docker_registries_to_login }}"

templates/docker-rotate

Lines changed: 0 additions & 7 deletions
This file was deleted.

templates/docker-rotate.j2

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
#!/bin/sh
2+
3+
{% if docker_rotate_images_arguments | bool %}
4+
docker-rotate images {{ docker_rotate_images_arguments }}
5+
{% endif %}
6+
7+
{% if docker_rotate_untagged_images %}
8+
docker-rotate untagged-images
9+
{% endif %}
10+
11+
{% if docker_rotate_exited_containers != "" %}
12+
docker-rotate containers --exited {{ docker_rotate_exited_containers }}
13+
{% endif %}
14+
15+
{% if docker_rotate_created_containers != "" %}
16+
docker-rotate containers --created {{ docker_rotate_created_containers }}
17+
{% endif %}

templates/docker.j2

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,9 @@
11
# Docker Upstart and SysVinit configuration file
22

3-
# Customize location of Docker binary (especially for development testing).
4-
#DOCKER="/usr/local/bin/docker"
5-
6-
# Use DOCKER_OPTS to modify the daemon startup options.
7-
#DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4"
8-
{% if docker_opts is defined %}
9-
DOCKER_OPTS="{{ docker_opts }}"
10-
{% endif %}
11-
# If you need Docker to use an HTTP proxy, it can also be specified here.
12-
#export http_proxy="http://127.0.0.1:3128/"
13-
14-
# This is also a handy place to tweak where Docker's temporary files go.
15-
#export TMPDIR="/mnt/bigdrive/docker-tmp"
3+
DOCKER_OPTS="
4+
{% if docker_opts is defined %}
5+
{{ docker_opts }}
6+
{%- endif %}
7+
{% if docker_version | version_compare('1.8', '>=') -%}
8+
--log-opt max-size={{ docker_log_rotate_max_size }} --log-opt max-file={{ docker_log_rotate_count }}
9+
{%- endif %}"

templates/logrotate.conf.j2

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/var/lib/docker/containers/*/*-json.log {
2-
{{ docker_log_rotate_interval }}
2+
size {{ docker_log_rotate_max_size }}
33
rotate {{ docker_log_rotate_count }}
44
missingok
55
compress

templates/requirements.txt.j2

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
docker-py{{ '' if docker_py_version == 'latest' else ('==' + docker_py_version) }}
2+
dockerrotate=={{ docker_rotate_version }}
3+
{% if docker_compose_version is not none and docker_compose_version != '' %}
4+
docker-compose{{ '' if docker_compose_version == 'latest' else ('==' + docker_compose_version) }}
5+
{% endif %}

0 commit comments

Comments
 (0)