Skip to content

Commit

Permalink
Catch MissingDatasetTypeError in making quantum graph
Browse files Browse the repository at this point in the history
If a dataset type isn't known to the local repo, the quantum
graph building now fails with MissingDatasetTypeError. It
was ignored in the past. This scenario is possible in cases
such as a fresh local repo is used and the incoming image
doesn't have matching templates. We want it to fall back to
try the next pipeline. If the repo knows the template's
dataset type but no matching templates exist, it still
generate an empty quantum graph.
  • Loading branch information
hsinfang committed Jan 24, 2024
1 parent bedacc8 commit c8311b0
Showing 1 changed file with 7 additions and 2 deletions.
9 changes: 7 additions & 2 deletions python/activator/middleware_interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
import lsst.ctrl.mpexec
from lsst.ctrl.mpexec import SeparablePipelineExecutor, SingleQuantumExecutor, MPGraphExecutor
from lsst.daf.butler import Butler, CollectionType, Timespan
from lsst.daf.butler.registry import MissingDatasetTypeError
import lsst.dax.apdb
import lsst.geom
from lsst.meas.algorithms.htmIndexer import HtmIndexer
Expand Down Expand Up @@ -862,10 +863,14 @@ def run_pipeline(self, exposure_ids: set[int]) -> None:
skip_existing_in=None,
task_factory=factory,
)
qgraph = executor.make_quantum_graph(pipeline, where=where)
try:
qgraph = executor.make_quantum_graph(pipeline, where=where)
except MissingDatasetTypeError as e:
_log.error(f"Building quantum graph for {pipeline_file} failed ", exc_info=e)
continue
if len(qgraph) == 0:
# Diagnostic logs are the responsibility of GraphBuilder.
_log.error(f"Could not build quantum graph for {pipeline_file}; "
_log.error(f"Empty quantum graph for {pipeline_file}; "
"see previous logs for details.")
continue
# Past this point, partial execution creates datasets.
Expand Down

0 comments on commit c8311b0

Please sign in to comment.