[Automatic Import] Reduce the number of categorization errors #198326
Labels
bug
Fixes for quality problems that affect the customer experience
Feature:AutomaticImport
Team:Security-Scalability
Team label for Security Integrations Scalability Team
Context
Integrations like Postgres are quite complex and sometimes the categorization process does not complete within the allowed 2 minutes.
I think we can focus on improving the categorization process to reduce the number of failed generations.
We currently ask the LLM to generate the ingest pipeline directly. This produces incorrect combinations of category + type, possibly because the LLM is not smart enough to notice that (the ingest pipeline is non-local):
Suggestion
We can instead ask it to produce a list of conditions and pairs of category + type, so that those two values are located nearby:
would be much easier for the LLM to notice.
The text was updated successfully, but these errors were encountered: