[Automatic Import] Reduce the number of categorization errors

[ilyannn]

Context

Integrations like Postgres are quite complex and sometimes the categorization process does not complete within the allowed 2 minutes.

I think we can focus on improving the categorization process to reduce the number of failed generations.

We currently ask the LLM to generate the ingest pipeline directly. This produces incorrect combinations of category + type, possibly because the LLM is not smart enough to notice that (the ingest pipeline is non-local):

[
  {
    "field": "event.category",
    "value": [
      "database"
    ]
  },
  {
    "field": "event.type",
    "value": [
      "info"
    ]
  },
  {
    "field": "event.type",
    "value": [
      "start"
    ],
    "if": "ctx.message?.contains('starting PostgreSQL')"
  },
...

Suggestion

We can instead ask it to produce a list of conditions and pairs of category + type, so that those two values are located nearby:

- if: "ctx.message?.contains('starting PostgreSQL')"
- classify: database + start

would be much easier for the LLM to notice.