ceph-mon charm runs into error when using focal-ussuri

[pponnuvel]

Reproducer:

cd stsstack-bundles/ceph
./generate-bundle.sh -n ceph-octopus --no-openstack --run

And ceph-mon will end up in error state. Log says:

charmhelpers.fetch.sourceconfigerror: unsupported cloud: source option focal-ussuri

Changing the source option to distro "fixes" it.

[pponnuvel]

Full backtrace:

2023-05-17 14:39:27 INFO juju.worker.uniter.storage resolver.go:127 initial storage attachments ready
2023-05-17 14:39:27 INFO juju.worker.uniter resolver.go:159 found queued "install" hook
2023-05-17 14:40:43 INFO unit.ceph-mon/0.juju-log server.go:316 Installing ['ubuntu-cloud-keyring'] with options: ['--option=Dpkg::Options::=--force-confold']
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60 Traceback (most recent call last):
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60   File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/install.real", line 1351, in <module>
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60     hooks.execute(sys.argv)
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60   File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/charmhelpers/core/hookenv.py", line 962, in execute
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60     self._hooks[hook_name]()
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60   File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/charmhelpers/contrib/hardening/harden.py", line 93, in _harden_inner2
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60     return f(*args, **kwargs)
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60   File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/install.real", line 164, in install
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60     add_source(config('source'), config('key'))
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60   File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/charmhelpers/fetch/ubuntu.py", line 707, in add_source
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60     fn(*m.groups())
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60   File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/charmhelpers/fetch/ubuntu.py", line 804, in _add_cloud_distro_check
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60     _add_cloud_pocket("{}-{}".format(cloud_archive_release, openstack_release))
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60   File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/charmhelpers/fetch/ubuntu.py", line 764, in _add_cloud_pocket
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60     raise SourceConfigError(
2023-05-17 14:40:47 WARNING unit.ceph-mon/0.install logger.go:60 charmhelpers.fetch.SourceConfigError: Unsupported cloud: source option focal-ussuri
2023-05-17 14:40:48 ERROR juju.worker.uniter.operation runhook.go:153 hook "install" (via explicit, bespoke hook script) failed: exit status 1

Here's simple bundle that, when juju deployed, would reproduce the issue:

loglevel:                   &loglevel                  1
source:                     &source                    
num_mon_units:              &num_mon_units             1

series: focal
applications:
  ceph-mon:
    charm: ch:ceph-mon
    channel: octopus/edge
    num_units: *num_mon_units
    constraints: mem=2G
    options:
      source: *source
      loglevel: *loglevel
      monitor-count: *num_mon_units
      monitor-secret: AQCXrnZQwI7KGBAAiPofmKEXKxu5bUzoYLVkbQ==
      expected-osd-count: 3
  ceph-osd:
    charm: ch:ceph-osd
    channel: octopus/edge
    num_units: 3
    constraints: mem=1G
    options:
      source: *source
      loglevel: *loglevel
      osd-devices: ''  # must be empty string when using juju storage
    storage:
      osd-devices: cinder,10G,1
relations:
  - [ ceph-mon, ceph-osd ]

[marino-mrc]

Hello,
I tried to mitigate this with:

juju config ceph-mon source=distro

It works but I'm not sure how this impact other components in big environments (openstack + ceph). Do you have any other workaround for it?

Thanks,
Marco

[nicolasbock]

This could be a regression in the generate script. But the underlying problem seems to be that the ceph packages don't exist in the focal Ussuri UCA.

[pponnuvel]

This could be a regression in the generate script. But the underlying problem seems to be that the ceph packages don't exist in the focal Ussuri UCA.

Ceph appears to exist in UCA (Ussuri):

 ceph | 15.2.17-0ubuntu0.20.04.1~cloud0  | ussuri           | bionic-updates  | source
 ceph | 15.2.17-0ubuntu0.20.04.4~cloud0  | ussuri-proposed  | bionic-proposed | source

Downloading confirms it:

  pull-uca-source ceph ussuri
Found ceph 15.2.17-0ubuntu0.20.04.4~cloud0 in bionic
Downloading ceph_15.2.17-0ubuntu0.20.04.4~cloud0.dsc from launchpadlibrarian.net (0.008 MiB)
[=====================================================>]100%
Public key not found, could not verify signature
Downloading ceph_15.2.17.orig.tar.xz from launchpadlibrarian.net (94.503 MiB)
[=====================================================>]100%
Downloading ceph_15.2.17-0ubuntu0.20.04.4~cloud0.debian.tar.xz from launchpadlibrarian.net (0.118 MiB)
[=====================================================>]100%

[nicolasbock]

Isn't that in Bionic though? I think the focal package should come from the standard repositories.

[KellenRenshaw]

Correct, Focal distro packages are Octopus/Ussuri. The charm needs a fix to not try a non-existent cloud archive.

[KellenRenshaw]

Opened https://bugs.launchpad.net/charm-ceph-mon/+bug/2045430 on the ceph-mon charm