Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement asynchronous publisher #1050

Merged
merged 13 commits into from
Jan 24, 2024
Merged

Implement asynchronous publisher #1050

merged 13 commits into from
Jan 24, 2024

Conversation

jpmckinney
Copy link
Member

@jpmckinney jpmckinney commented Jan 24, 2024

closes #1033

…isherProcessAPI2 to 500, now that KingfisherProcessAPI is removed.
…bbitMQ restarts.

test: Simplify test_spider_closed to test_spider_closed_error, as the success case is covered by test_spider_opened.
test: Remove "raises" logic from test_item_scraped, as this works only for a blocking basic_publish call.
Revert "feat: Use the asyncio reactor, to be able to use yapw async clients" 1884445
Revert "test: Set the reactor correctly" 442bbf1

Note: kingfisher_process_items_sent_rabbit and kingfisher_process_items_failed_rabbit stats are no longer tracked,
as publishing is not expected to error.
…twisted.internet.error.ReactorAlreadyRunning"
…tension waits for the new connection to be established
@jpmckinney jpmckinney merged commit b0ae329 into main Jan 24, 2024
5 of 7 checks passed
@jpmckinney jpmckinney deleted the async-publisher branch January 24, 2024 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use async publisher, instead of blocking connection
1 participant