Generally speaking, I'm against performing major OS upgrades on my Linux servers; there are often little things that get broken, or configurations gone awry, when you attempt an upgrade... and part of the point of automation (or striving towards a 12-factor app) is that you don't 'upgrade'—you destroy and rebuild with a newer version.
But, there are still cases where you have legacy servers running one little task that you haven't yet automated entirely, or that have data on them that is not yet stored in a way where you can tear down the server and build a new replacement. In these cases, assuming you've already done a canary upgrade on a similar but disposable server (to make sure there are no major gotchas), it may be the lesser of two evils to use something like Ubuntu's do-release-upgrade.
Since Ubuntu 12.04 LTS was end-of-lifed earlier this year, and I had about a dozen servers for one of my services that were still running 12.04, I decided to bite the bullet (I've been procrastinating replacing these servers a while!), and perform the in-place upgrade. I also ensured that a fresh backup was taken as part of the upgrade process (using my simple geerlingguy.backup role for Ansible.
Here's the playbook I used—note that it will only perform a release upgrade on Ubuntu 12.04 LTS servers. If you want to upgrade from 14.04 to 16.04, then you'd need to change a few numbers in the playbook to '14.04' instead of '12.04':
---
- hosts: legacyservers
gather_facts: yes
become: yes
tasks:
# Use a block to perform tasks conditionally—only if running Ubuntu 12.04.
- block:
- debug:
msg: 'This server is running Ubuntu 12.04 LTS and will be upgraded to 14.04 LTS.'
# Now would be a good time to take a backup if you can trigger an
# automated backup!
- name: Remove the EOL message of the day if one exists.
file:
path: "{{ item }}"
state: absent
with_items:
- /etc/update-motd.d/99-esm
- /run/motd.dynamic
- name: Upgrade all packages to the latest version
apt: update_cache=yes upgrade=full
- name: Ensure update-manager-core is installed.
apt: name=update-manager-core state=present
- name: Run do-release-upgrade non-interactively.
command: do-release-upgrade -f DistUpgradeViewNonInteractive
- name: Reboot the server.
reboot:
when:
- ansible_distribution == 'Ubuntu'
- ansible_distribution_version == '12.04'
# After the playbook is finished, it's a good idea to confirm all the servers
# are actually upgraded. Run something like:
# ansible [group] -a "lsb_release -a"This idempotent* playbook does the following:
- Prints a message indicating an inventory server will be upgraded.
- Clears out the pesky 'Ubuntu 12.04 has reached end-of-life' message of the day (MOTD) so you don't see it next time you log into one of these servers.
- Upgrades everything to the latest package versions.
- Ensures
update-manager-coreis installed sodo-release-upgradecan be run. - Runs the release upgrade, reboots the server, and waits for the server to respond on port 22 (SSH) before completing.
After you run a playbook like this, you may want to manually verify the servers are all upgraded. You can use an ad-hoc command to do that:
ansible [group] -a "lsb_release -a"
Technically, you could add a task in the playbook that performs this confirmation task... but whatever way works, works!
* Idempotent = you can run the playbook multiple times and once it gets things in a certain state (e.g. 12.04 hosts upgraded to 14.04), it will change nothing else and report 'ok' instead of 'changed' during ansible-playbook runs.
Comments
How did you deal with the questions asked during the upgrade? Most of the time I answered to default 'N' but when running the non-interactive upgrade even local on the server it still asked for those confirmations. I was upgrading from 16.04 to 18.04 so not sure if you ran into the same issues during your upgrade.
For preventing the prompts in do-release-upgrade, something like this should do it:
1. create file 'apt_conf_local' under files in your ansible source, with dpkg options as content
e.g.
DPkg::options { "--force-confdef"; "--force-confnew"; }
2. add a tasks (before and after the upgrade respectively)
- name: set dpkg options
copy:
src: apt_conf_local
dest: /etc/apt/apt.conf.d/local
mode: 0644
#insert upgrade tasks etc here
- name: cleanup dpkg options
file:
path: /etc/apt/apt.conf.d/local
state: absent
Hi,
The above process is worked if it will not prompt questions during upgrade process. I will be waiting for your answer.
This playbook is really awesome. i used it to version all my libvirt / kvm virtualization servers Ubuntu-18.04 to Ubuntu-20-04. Many thanks for this work.