Our Ubuntu 14.04 cluster has been running Ceph Jewel without issue for about the last 9 months. During that time period the Ceph team released the latest LTS version called Luminous. Given that Jewel is slated for EOL at some point int he next 2 or 3 months, I thought it was a good time to once again upgrade the cluster.
Before I started working on upgrading the version of Ceph on the cluster I decided to go ahead and upgrade the version of Ubuntu from 14.04 to 16.04. The process for me was a relatively simple one, I used the following single set of commands on each node:
apt-get update; apt-get upgrade; apt-get dist-upgrade ; apt-get install update-manager-core; do-release-upgrade
After the upgrade the process for restarting Ceph daemons will have changed slightly due to the fact that Ubuntu 16.04 uses systemd instead of upstart. For example you will have to use the following commands to restart each of the services going forward:
systemctl restart ceph-mon.target
systemctl restart ceph-osd.target
systemctl restart ceph-mgr.target
After the Ubuntu upgrade I decided to start upgrading the version of Ceph on the cluster.
Using ‘ceph-deploy’ I issued the following commands on each node:
Monitor nodes first:
ceph-deploy install --release luminous hqceph1 hqceph2 hqceph3 (from the admin node)
systemctl restart ceph-mon.target (locally on each server)
OSD nodes second (for example):
Set ‘noout’ so your data does not try to rebalance during the OSD restarts:
ceph osd set noout
ceph-deploy install --release luminous hqosd1 hqosd2 hqosd3Â (from admin node)
systemctl restart ceph-osd.target (locally on each server)
Next you should do the same for any RGW nodes that you might have.
Finally repeat the process for any other client nodes (librbd, KRBD, etc).
Don’t forget to unset noout before you finish:
ceph osd unset noout (from the admin node)
This release also introduced a new daemon called the manager daemon or ‘mgr’. You can learn more about this new process here:  http://docs.ceph.com/docs/mimic/mgr/
I was able to install the mgr daemon on two of my existing nodes (one as a active and the second as the standyby) using the following commands:
ceph-deploy mgr create hqceph2 (from the admin node)
ceph-deploy mgr create hqceph3 (from the admin node)