The job definition does not exist anymore and zuul config reports error
because of this, so let's get rid of this from the check queue (gate
queue does not even have it).

Change-Id: I0b211b7c8fcfebeaf9f1bc9bbee0c6527faa53d9

py39 jobs (on ubuntu-focal) started to fail due to recent virtualenv
release (20.26.4 that bundles setuptools 74.1.2) on Yoga, because
we have 'packaging==21.3' in this branch that is not compatible
with the new setuptools [1].

setuptools is bundled in virtualenv, so it has to be capped via the
virtualenv package. tox also needed to be capped (<4) as gate uses
tox 3.28.0, but with capping virtualenv we pull in latest tox as well,
which would cause other errors.

[1] pypa/setuptools#4483

Change-Id: I668557933c51123ce84275a0c718b6e79b3df174

This adds a safety check and detection mechanism to the
tools/test_format_inspector.py utility for verifying those features
outside of glance.

Change-Id: I447e7e51315472f8fa6013d4c4852f54c1e0c43d
(cherry picked from commit b27040b87e43ff4acfb6870ef0d13d54e5ca5caa)
(cherry picked from commit b394ef00c3426771092d099cf1d96077bfa4b919)
(cherry picked from commit aa12d39b453068d9ee8367591eb60d4a15cddece)
(cherry picked from commit a1c94b4)
(cherry picked from commit a5ec69b)

…ned/yoga

We have to address two issues simultaneously:

1. handle multiple cirros images

   The change[1] configures multiple images for the devstack-tempest
   job and all the jobs deriving from it. This causes problem with
   the current post-check-metadata-injection.yaml playbook since
   we only expect one cirros image and perform steps accordingly.
   This change modifies the playbook to handle multiple cirros
   images.
   Failure in the glance-multistore-cinder-import job can be
   seen here[2].

   [1] https://review.opendev.org/c/openstack/tempest/+/831018
   [2] https://zuul.opendev.org/t/openstack/build/68a4d3ec6ce04c87b21d73a333f5b5cd/log/job-output.txt#23161

   This was change I70b4a970d7e1.

2. change the the job config to address resource constraints
   * partial backport of I073216d1bbdd to change swap size
     (we don't include the change from that patch that was needed
     to address changes in Bobcat-era devstack)
   * backporting Ieb96eb6ceb6f to change tempest concurrency

Remove openstack-tox-functional-py36-fips as there is no
job definition for it.

Change-Id: I6cd5b8bbf39f0cfae466f0800faea5a74e960e69
(cherry picked from commit 065e2d7)
(cherry picked from commit 28b05a7)

VMDK:
When parsing a VMDK file to calculate its size, the format_inspector
determines the location of the Descriptor section by reading two
uint64 from the headers of the file and uses them to create the
descriptor CaptureRegion.

It would be possible to craft a VMDK file that commands the
format_inspector to create a very big CaptureRegion, thus exhausting
resources on the glance-api process.

This patch binds the beginning of the descriptor to 0x200 and limits
the size of the CaptureRegion to 1MB, similar to how the VMDK
descriptor is parsed by qemu.

VHDX:
It is a bit more involved, but similar: when looking for the
VIRTUAL_DISK_SIZE metadata, the format_inspector was creating an
unbounded CaptureRegion.

In the same way as it seems to be done in Qemu, we now limit the upper
bound of this CaptureRegion.

Closes-Bug: #2006490
Change-Id: I3ec5a33df20e1cfb6673f4ff1c7c91aacd065532
(cherry picked from commit d4d33ee)
(cherry picked from commit 06a1820)

Stream optimized VMDKs are also monolithic disks images, and start
with the same sparse extend header as normal monolithic sparse files,
so we can parse the virtual disk size in the same manner.

See "VMware Virtual Disks Virtual Disk Format 1.1" p. 17.
> Header and Footer
> The header and the footer are both described by the same SparseExtentHeader
> structure shown in Hosted Sparse Extent Header on page 8.

Closes-Bug: #2052291
Change-Id: I7d63951ff080dc699b8d11babc0a5998d90621e4
Co-Authored-By: Rajiv Mucheli <[email protected]>
(cherry picked from commit 5e7e6bf)
(cherry picked from commit 9d45e8d4b87e992be23974a831811bff563ce721)
(cherry picked from commit ef22a74)
(cherry picked from commit bcfc1e4)