Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chunk through high-volume produce requests #978

Closed
shouples opened this issue Jan 27, 2025 · 1 comment
Closed

Chunk through high-volume produce requests #978

shouples opened this issue Jan 27, 2025 · 1 comment
Assignees
Labels
optimization Make something more efficient

Comments

@shouples
Copy link
Contributor

Going above 2500 seems to cause some blockage:
Image

We should also try to increment the existing progress notification:

vscode.window.withProgress(
{
location: vscode.ProgressLocation.Notification,
title: `Producing ${contents.length} message${contents.length > 1 ? "s" : ""} to topic "${topic.name}"...`,
cancellable: false,
},
async () => {
// TODO: chunk through this and increment progress?
await produceMessageBatch(contents, topic);
},
);
}
}
async function produceMessageBatch(contents: any[], topic: KafkaTopic) {
const plural = contents.length > 1 ? "s" : "";
const promises: Promise<ProduceResult>[] = contents.map((content) =>
produceMessage(content, topic),
);
const produceResults: ProduceResult[] = await Promise.all(promises);

@shouples shouples added the optimization Make something more efficient label Jan 27, 2025
@shouples shouples self-assigned this Jan 27, 2025
@shouples
Copy link
Contributor Author

shouples commented Feb 4, 2025

Done in #992

@shouples shouples closed this as completed Feb 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
optimization Make something more efficient
Projects
None yet
Development

No branches or pull requests

1 participant