Ansible: Update Servers to the Latest and Reboot

This is for Debian/Ubuntu flavored systems.

Keep a single server up to date is easy, but updating multiple servers at once, you need tools like Ansible. For each server, here is a list of basic steps:

  1. Check if there are packages available to be upgraded
  2. Upgrade all packages to the latest version
  3. Check if a reboot is required
  4. Reboot the server

When we log into the remote server, we might see the message showing the number of packages can be updated. The message is generated by:

1
2
3
4
$ sudo /usr/lib/update-notifier/update-motd-updates-available 
25 packages can be updated.
18 updates are security updates.

And it is available at:

1
2
3
4
$ cat /var/lib/update-notifier/updates-available                                       
25 packages can be updated.
18 updates are security updates.

We don’t need that detailed information, we just simply want to know if there are update available.

Shell script /usr/lib/update-notifier/apt-check shows any pending updates:

1
2
$ /usr/lib/update-notifier/apt-check
25;18

To list all the packages instead of simple packages;security format:

1
$ /usr/lib/update-notifier/apt-check --package-names

--package-names option will write data to stderr instead of stdout. If there are no packages needed to be installed, then the stderr should be empty.

If there are packages to be installed or upgraded. Ansible has the apt module to manage them in Debian/Ubuntu based systems.

1
2
3
4
5
6
7
- name: Check if there are packages available to be installed/upgraded
  command: /usr/lib/update-notifier/apt-check --package-names
  register: packages
- name: Upgrade all packages to the latest version
  apt: update_cache=yes upgrade=dist
  when: packages.stderr != ""

When a reboot is required in Debian/Ubuntu systems, the file /var/run/reboot-required will be present.

1
$ test -e /var/run/reboot-required && echo 'Reboot required'

Therefore, we can simply check the file, and reboot the system.

The difference is also captured here:

1
2
3
4
5
$ uname -r
3.2.0-67-virtual
$ cat /var/run/reboot-required.pkgs 
linux-image-3.2.0-68-virtual
linux-base

Therefore, these two packages are present if requiring a reboot:

1
2
/var/run/reboot-required
/var/run/reboot-required.pkgs

After reboot, these two files will be removed. Both files are updated by this shell script:

1
/usr/share/update-notifier/notify-reboot-required

So, the playbook can be written as:

1
2
3
- name: Check if a reboot is required
  register: file
  stat: path=/var/run/reboot-required get_md5=no

The registered file stat looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
{
  "atime": 1410212575.8964653,
  "ctime": 1410212506.3324652,
  "dev": 15,
  "exists": true,
  "gid": 0,
  "inode": 2336609,
  "isblk": false,
  "ischr": false,
  "isdir": false,
  "isfifo": false,
  "isgid": false,
  "islnk": false,
  "isreg": true,
  "issock": false,
  "isuid": false,
  "mode": "0644",
  "mtime": 1410212506.3324652,
  "nlink": 1,
  "pw_name": "root",
  "rgrp": true,
  "roth": true,
  "rusr": true,
  "size": 32,
  "uid": 0,
  "wgrp": false,
  "woth": false,
  "wusr": true,
  "xgrp": false,
  "xoth": false,
  "xusr": false
}

We can see the file exists ("exists": true), hence reboot is needed:

1
2
3
- name: Reboot the server
  command: /sbin/reboot
  when: file.stat.exists == true

In summary, adding the reboot tasks and putting them all together in an Ansible playbook task file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# Upgrade Debian/Ubuntu based systems and reboot if necessary.
---
- name: Check if there are packages available to be installed/upgraded
  command: /usr/lib/update-notifier/apt-check --package-names
  register: packages
- name: Upgrade all packages to the latest version
  apt: update_cache=yes upgrade=dist
  when: packages.stderr != ""
- name: Check if a reboot is required
  register: file
  stat: path=/var/run/reboot-required get_md5=no
- name: Reboot the server
  command: /sbin/reboot
  when: file.stat.exists == true

The only problem is that when the reboot is issued, the command exits, and the system is yet to go down for reboot right away. Any tasks after that will continue to be run. Therefore, it is not a good idea to add more tasks after the reboot task.