FIX #19386 & #19388: Fixing Data Insights index mapping

[IceS2]

This PR aims to solve three things:

1. First and foremost, it aims to make the Data Insights indexes more reliable, by using more static mapping, in order to rely less on the dynamic mapping.
2. Meanwhile it also exposes a command for OpenMetadataOps to recreate and backfill Data Insights indexes.
3. Finally, it aims to reduce the amount of data that we are indexing, by reducing the amount of attributes in the data insights documents.

Fix #19386
Fix #19388

Type of change:

• Improvement

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes <issue-number>: <short explanation>
• I have commented on my code, particularly in hard-to-understand areas.
• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

[TeddyCr]

Basically if we want to add new fields/metrics (or support for a new entity) we'll need to add those to this file, is that correct?

[IceS2]

In order to support a new entity actually we have a list on the Java Application itself (It'd be a good idea to remove it and basically support based on the list)

But yes, in order to add any new fields, we'd need to add it there. That way we'd control the amount of data indexed and also we're fetching the mapping that we define in our not-data-insights indexes to reduce the amount of dynamic mapping

[mohityadav766]

we should allow the backfill configuration here to be supplied if needed ,
else only way would be to use UI to set it and then run here if it needs to be run with some different config

[IceS2]

It is already there:

  @Option(
          names = {"--start-date"},
          description = "Start Date to backfill from.")
      String startDate,
  @Option(
          names = {"--end-date"},
          description = "End Date to backfill to.")
      String endDate) {

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

Changes have been cherry-picked to the 1.6.3 branch.