CI qubit selection tests failure

[ElePT]

Environment

• Qiskit version: 1.2.2
• Python version: 3.8 and 3.11
• Operating system: ubuntu, windows and mac

What is happening?

After some time without activity, CI qubit selection tests seem to yield different results. This may be due to changes in the fake backend used (FakeSherbrooke) or the qiskit SDK.

How can we reproduce the issue?

Error trace:

{1} test.test_qubit_selection.TestQubitSelection.test_qubit_selection_v1_v2 [0.032841s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/runner/work/qopt-best-practices/qopt-best-practices/test/test_qubit_selection.py", line 63, in test_qubit_selection_v1_v2
    self.assertEqual(set(path), set(expected_path))

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 873, in assertEqual
    assertion_func(first, second, msg=msg)

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 1133, in assertSetEqual
    self.fail(self._formatMessage(msg, standardMsg))

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 703, in fail
    raise self.failureException(msg)

    AssertionError: Items in the first set but not the second:
9
15
18
21
23
Items in the second set but not the first:
1
2
4
7
10

{1} test.test_sat_mapping.TestSwapStrategies.test_deficient_strategy [0.023765s] ... ok
{1} test.test_sat_mapping.TestSwapStrategies.test_remap_graph_with_sat [0.078331s] ... ok
/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/qiskit/qobj/pulse_qobj.py:576: DeprecationWarning: The class ``qiskit.qobj.pulse_qobj.PulseLibraryItem`` is deprecated as of qiskit 1.2. It will be removed in the 2.0 release. The `Qobj` class and related functionality are part of the deprecated `BackendV1` workflow,  and no longer necessary for `BackendV2`. If a user workflow requires `Qobj` it likely relies on deprecated functionality and should be updated to use `BackendV2`.
  return cls(**data)
/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/qiskit/qobj/pulse_qobj.py:273: DeprecationWarning: The class ``qiskit.qobj.pulse_qobj.PulseQobjInstruction`` is deprecated as of qiskit 1.2. It will be removed in the 2.0 release. The `Qobj` class and related functionality are part of the deprecated `BackendV1` workflow,  and no longer necessary for `BackendV2`. If a user workflow requires `Qobj` it likely relies on deprecated functionality and should be updated to use `BackendV2`.
  return cls(data["name"], data["t0"], **in_data)
/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/qiskit/qobj/pulse_qobj.py:576: DeprecationWarning: The class ``qiskit.qobj.pulse_qobj.PulseLibraryItem`` is deprecated as of qiskit 1.2. It will be removed in the 2.0 release. The `Qobj` class and related functionality are part of the deprecated `BackendV1` workflow,  and no longer necessary for `BackendV2`. If a user workflow requires `Qobj` it likely relies on deprecated functionality and should be updated to use `BackendV2`.
  return cls(**data)
/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/qiskit/qobj/pulse_qobj.py:273: DeprecationWarning: The class ``qiskit.qobj.pulse_qobj.PulseQobjInstruction`` is deprecated as of qiskit 1.2. It will be removed in the 2.0 release. The `Qobj` class and related functionality are part of the deprecated `BackendV1` workflow,  and no longer necessary for `BackendV2`. If a user workflow requires `Qobj` it likely relies on deprecated functionality and should be updated to use `BackendV2`.
  return cls(data["name"], data["t0"], **in_data)
/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/threading.py:986: ResourceWarning: unclosed file <_io.FileIO name=7 mode='rb' closefd=True>
  del self._target, self._args, self._kwargs
ResourceWarning: Enable tracemalloc to get the object allocation traceback
{3} test.test_qubit_selection.TestQubitSelection.test_find_lines [0.221714s] ... ok
{1} test.test_swap_strategies.TestSwapStrategies.test_qaoa_circuit [0.123[53](https://github.com/qiskit-community/qopt-best-practices/actions/runs/11164635668/job/31034465866?pr=34#step:7:55)7s] ... ok
{0} test.test_qubit_selection.TestQubitSelection.test_qubit_selection [0.505207s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/runner/work/qopt-best-practices/qopt-best-practices/test/test_qubit_selection.py", line 53, in test_qubit_selection
    self.assertEqual(set(path), set(expected_path))

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 873, in assertEqual
    assertion_func(first, second, msg=msg)

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 1133, in assertSetEqual
    self.fail(self._formatMessage(msg, standardMsg))

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 703, in fail
    raise self.failureException(msg)

    AssertionError: Items in the first set but not the second:
33
39
/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/threading.py:986: ResourceWarning: unclosed file <_io.FileIO name=6 mode='rb' closefd=True>
  del self._target, self._args, self._kwargs
ResourceWarning: Enable tracemalloc to get the object allocation traceback
40
72
41
81
53
60
61
62
Items in the second set but not the first:
68
69
70
74
45
46
47
48
49
[55](https://github.com/qiskit-community/qopt-best-practices/actions/runs/11164635668/job/31034465866?pr=34#step:7:57)

{0} test.test_sat_mapping.TestSwapStrategies.test_find_initial_mappings [0.014376s] ... ok
{0} test.test_sat_mapping.TestSwapStrategies.test_unable_to_remap [0.0026[56](https://github.com/qiskit-community/qopt-best-practices/actions/runs/11164635668/job/31034465866?pr=34#step:7:58)s] ... ok
/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/threading.py:986: ResourceWarning: unclosed file <_io.FileIO name=5 mode='rb' closefd=True>
  del self._target, self._args, self._kwargs
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/stestr/commands/load.py:350: ResourceWarning: unclosed file <_io.FileIO name=8 mode='rb' closefd=True>
  case.run(result)
ResourceWarning: Enable tracemalloc to get the object allocation traceback

==============================
Failed 2 tests - output below:
==============================

test.test_qubit_selection.TestQubitSelection.test_qubit_selection_v1_v2
-----------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/runner/work/qopt-best-practices/qopt-best-practices/test/test_qubit_selection.py", line 63, in test_qubit_selection_v1_v2
    self.assertEqual(set(path), set(expected_path))

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 873, in assertEqual
    assertion_func(first, second, msg=msg)

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 1133, in assertSetEqual
    self.fail(self._formatMessage(msg, standardMsg))

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 703, in fail
    raise self.failureException(msg)

    AssertionError: Items in the first set but not the second:
9
15
18
21
23
Items in the second set but not the first:
1
2
4
7
10


test.test_qubit_selection.TestQubitSelection.test_qubit_selection
-----------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/runner/work/qopt-best-practices/qopt-best-practices/test/test_qubit_selection.py", line 53, in test_qubit_selection
    self.assertEqual(set(path), set(expected_path))

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 873, in assertEqual
    assertion_func(first, second, msg=msg)

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 1133, in assertSetEqual
    self.fail(self._formatMessage(msg, standardMsg))

      File "/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/unittest/case.py", line 703, in fail
    raise self.failureException(msg)

    AssertionError: Items in the first set but not the second:
33
39
40
72
41
81
53
[60](https://github.com/qiskit-community/qopt-best-practices/actions/runs/11164635668/job/31034465866?pr=34#step:7:62)
[61](https://github.com/qiskit-community/qopt-best-practices/actions/runs/11164635668/job/31034465866?pr=34#step:7:63)
62
Items in the second set but not the first:
68
69
70
74
45
46
47
48
49
55

What should happen?

Test results shouldn't have changed. Note that we are updating the tests to the new output but this doesn't mean we have addressed the root cause.

Any suggestions?

Seems like there haven't been major updates to the fake backends beyond adding control flow instructions in the past 6 months, so this might point to a Qiskit-related issue. TBD