Docker and systemd, getting rid of dreaded 'Failed to connect to bus' error

The following error has been the bane of my existence for the past few months:

TASK [geerlingguy.containerd : Ensure containerd is started and enabled at boot.] ***
fatal: [instance]: FAILED! => {
  "changed": false,
  "cmd": "/bin/systemctl",
  "msg": "Failed to connect to bus: No such file or directory",
  "rc": 1,
  "stderr": "Failed to connect to bus: No such file or directory",
  "stderr_lines": [
    "Failed to connect to bus: No such file or directory"
  ],
  "stdout": "",
  "stdout_lines": []
}

Since I use Molecule with my Ansible roles and playbooks to test them in identical CI environments both locally and in GitHub Actions, I can maintain an identical environment inside which tests are run. And many of my roles and playbooks need to test whether systemd services are configured and run correctly.

But Docker recently switched from cgroups v1 to cgroups v2, and that started this 'Failed to connect to bus' business—systemd relied on some configuration that was easy enough to add in the past: just run your containers with these options:

--privileged -v /sys/fs/cgroup:/sys/fs/cgroup:rw

But after the cgroups v2 upgrade, you either had to add the option "deprecatedCgroupv1": true to Docker's settings.json file, or add the command line option --cgroupns=host.

The trouble is, Molecule didn't have a way to pass the cgroupns option, so if I wanted to run things locally, I was stuck having to use the 'deprecated' cgroup v1 option. Until yesterday!

All I had to do to ensure systemd would work inside my containers is add the cgroupns_mode: host option in my molecule.yml file, for example:

platforms:
  - name: instance
    image: "geerlingguy/docker-debian11-ansible:latest"
    command: ""
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:rw
      - /var/lib/containerd
    cgroupns_mode: host  ## <-- This is the line I added
    privileged: true
    pre_build_image: true

For more information, check out these issues:

Further reading

ansible
molecule
docker
systemd
continuous integration

Comments

Hi Jeff. Thank you for sharing this. Do you happen to know how to make commands like 'localectl' and 'timedatectl' work on rockylinux containers? I'm trying to write an ansible role to setup the system locale, but when I try to execute these commands during a 'molecule test' I still get 'Failed to connect to bus'. Debian 11 and Ubuntu 22.04 are fine

Thank you Jeff!

Once I'd got on the latest version of molecule this worked for me as well!

Hi Jeff

I tried using racnhed-desktop (moby) and using docker with podman and then running container inside no luck.
Pulling image podman pull got me

ror: writing blob: adding layer with blob "sha256:802b00ed6f79f48e6a5f44ecbcaf43563d6077aaecb565eee1dfc615c0b18c00": Error processing tar file(exit status 1): Error while loading /: Permission denied

I must admit I am using M1 mac book and fedora image amd64 on arm64 it's kind slow but other stuff work ok.
Any ideas how to solve this.

What is the significance of having the /sys/fs/cgroup:/sys/fs/cgroup set to Read/Write? For a while now, I have been able to run systemd in containers created by Molecule using Read-Only. Today I did an update which included a Docker update, and systemd wouldn't start in them until /sys/fs/cgroup:/sys/fs/cgroup was changed to Read/Write.
Running Docker inside WSL2, if that's important.