Support different embedding types of model response

[zane-neo]

Description

Support different embedding types of model response

Related Issues

#1006

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.71%. Comparing base (c36ca15) to head (7426f2e).
Report is 1 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #1007      +/-   ##
============================================
- Coverage     81.74%   81.71%   -0.03%     
+ Complexity     2509     1255    -1254     
============================================
  Files           190       95      -95     
  Lines          8560     4282    -4278     
  Branches       1436      718     -718     
============================================
- Hits           6997     3499    -3498     
+ Misses         1006      506     -500     
+ Partials        557      277     -280     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[martin-gaievski]

Can you provide sample request/response for each of new embedding type? Is the type configurable on the client side (neural search in this case), or model is smart enough to switch between types? Any changes required from user to use new type(s), for instance do they need to recreated model connector and index mapping?

[zane-neo]

Can you provide sample request/response for each of new embedding type? Is the type configurable on the client side (neural search in this case), or model is smart enough to switch between types? Any changes required from user to use new type(s), for instance do they need to recreated model connector and index mapping?

Sample of different embedding types:

{
    "float": [
        [
          -0.03933716,
          -0.04534912,
          -0.088256836,
          -0.084350586,
          ...
        ]
    ],
    "binary": [
        [
          -127,
          90,
          103,
          ...
        ]
    ],
    "int8": [
        [
          -50,
          -58,
          -113,
          ...
        ]
    ],
    "ubinary": [
        [
          1,
          218,
          231,
          ...
        ]
    ],
    "uint8": [
        [
          78,
          70,
          15,
          ...
        ]
    ]
}

The type is configurable in model side, in the connector specifically. When user wants to switch to a different embedding type, user needs to recreate a new model with different embedding type in the connector, also user should do a reindex to ensure the embeddings are using the new embedding type result, in this case, a new index should created and after the reindex user can specify alias on original index to new one. We don't consider modifying index mapping as we can only add new field in existing index mapping, not able to modify an existing field.

[yuye-aws]

@zane-neo Plz resolve the conflicting files. The code looks good to me.

[martin-gaievski]

@zane-neo is this PR ready for review or you expect more changes?

[yuye-aws]

The code looks good to me. Nice PR!

[zane-neo]

The failure BWC tests are flaky, @martin-gaievski @heemin32 This PR is ready for review, thanks.

[zhichao-aws]

Overall the PR looks good to me. The bwc issue should be fixed after we 1. bump to 3.0.0-alpha1 and 2. consider loaded and deployed as valid model states.

I saw both this PR and #1141 are fixing the model state issue. We should confirm we're not making duplicate fixes. cc @martin-gaievski

[zane-neo]

Overall the PR looks good to me. The bwc issue should be fixed after we 1. bump to 3.0.0-alpha1 and 2. consider loaded and deployed as valid model states.

I saw both this PR and #1141 are fixing the model state issue. We should confirm we're not making duplicate fixes. cc @martin-gaievski

@martin-gaievski Do you have a plan when to merge #1141? If it's going to have more changes, maybe we can merge this PR first?

[zhichao-aws]

#1141 is merged now. We can merge the PR after rebase main

[yuye-aws]

@zane-neo Conflict files need to be resolved

[martin-gaievski]

resolved conflict in CHANGELOG, @zane-neo in case more changes will be required please make sure to pull latest version and rebase on main

[martin-gaievski]

looks good to me, thank you