Skip to content

Commit

Permalink
feat: Kingfisher Process now expects kingfisher_process_expected_file…
Browse files Browse the repository at this point in the history
…s_count to equal the number of non-error messages
  • Loading branch information
jpmckinney committed Nov 2, 2024
1 parent 1817651 commit fd0f540
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions kingfisher_scrapy/extensions/kingfisher_process_api2.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,8 +173,9 @@ def item_scraped(self, item, spider):
cb = functools.partial(self._when_ready, self.client.publish, data, self.routing_key)
methods.add_callback_threadsafe(self.client.connection, cb)

# WARNING! Kingfisher Process's API reads this value.
self.stats.inc_value("kingfisher_process_expected_files_count")
if not isinstance(item, FileError):
# WARNING! Kingfisher Process's API reads this value.
self.stats.inc_value("kingfisher_process_expected_files_count")

def disconnect_and_join(self):
"""Close the RabbitMQ connection and join the client's thread."""
Expand Down

0 comments on commit fd0f540

Please sign in to comment.