Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Connectors][GenAI] Inference Service Kibana connector #189027

Merged

Conversation

YulNaumenko
Copy link
Contributor

@YulNaumenko YulNaumenko commented Jul 24, 2024

Summary

Resolves #188043

This PR adds new connector which is define integration with Elastic Inference Endpoint via Inference APIs
The lifecycle of the Inference Endpoint are managed by the connector registered handlers:

  • preSaveHook - create new Inference Endpoint in the connector create mode (isEdit === false) and delete+create in the connector edit mode (isEdit === true)
  • postSaveHook - check if the connector SO was created/updated and if not removes Inference Endpoint from preSaveHook
  • postDeleteHook - delete Inference Endpoint if connector was deleted.

In the Kibana Stack Management Connectors, its represented with the new card (Technical preview badge):

Screenshot 2024-09-27 at 2 11 12 PM

To simplify the future integration with AI Assistants, the Connector consists from the two main UI parts: provider selector and required provider settings, which will be always displayed
Screenshot 2024-10-07 at 7 59 09 AM

and Additional options, which contains optional provider settings and Task Type configuration:

Screenshot 2024-10-07 at 8 00 15 AM

subActions corresponds to the different taskTypes Inference API supports. Each of the task type has its own Inference Perform params. Currently added:

  • completion & completionStream
  • rerank
  • text_embedding
  • sparse_embedding

Follow up work:

  1. Collapse/expand Additional options, when the connector flyout/modal has AI Assistant as a context (path through the extending context implementation on the connector framework level)
  2. Add support for additional params for Completion subAction to be able to path functions
  3. Add support for tokens usage Dashboard, when inference API will include the used tokens count in the response
  4. Add functionality and UX for migration from existing specific AI connectors to the Inference connector with proper provider and completion task
  5. Integrate Connector with the AI Assistants

@YulNaumenko YulNaumenko added Feature:Actions/ConnectorTypes Issues related to specific Connector Types on the Actions Framework 8.16 candidate v8.16.0 labels Jul 24, 2024
@YulNaumenko YulNaumenko self-assigned this Jul 24, 2024
Copy link
Contributor

A documentation preview will be available soon.

Request a new doc build by commenting
  • Rebuild this PR: run docs-build
  • Rebuild this PR and all Elastic docs: run docs-build rebuild

run docs-build is much faster than run docs-build rebuild. A rebuild should only be needed in rare situations.

If your PR continues to fail for an unknown reason, the doc build pipeline may be broken. Elastic employees can check the pipeline status here.

@serenachou

This comment was marked as resolved.

@daveyholler
Copy link
Contributor

Some feedback from me, thanks for the video demo

@serenachou is there a link to said video demo?

@serenachou
Copy link

@daveyholler it's in the slack thread I added you and Julian to : https://elastic.slack.com/archives/C07DCCPGB55/p1721799374980719

@leemthompo
Copy link
Contributor

For simplicity, once Serena's suggestions have been drafted, please re-ping and we can copyedit from there :)

@daveyholler

This comment was marked as resolved.

@YulNaumenko
Copy link
Contributor Author

The typography (labels and headings) is a little wonky in the flyout.

I cannot change it, because it's a part of the connectors framework and UX consistency.

  1. Your buttons (Save & Test, Save) will likely need to move into the flyout footer and use the primary blue.
    CleanShot 2024-07-25 at 13 46 50

Buttons are also inherited from the connectors framework and there is no way to manage it's styles.
I also cannot make the Connector details group, because current "Connector settings" also belongs to the framework.

@YulNaumenko

This comment was marked as resolved.

Copy link
Contributor

@leemthompo leemthompo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few copyedits per @serenachou's drafts

…rence-connector

# Conflicts:
#	docs/settings/alert-action-settings.asciidoc
#	oas_docs/output/kibana.serverless.yaml
#	oas_docs/output/kibana.yaml
#	x-pack/plugins/actions/docs/openapi/bundled.yaml
.github/CODEOWNERS Outdated Show resolved Hide resolved
@serenachou
Copy link

@YulNaumenko will you have an updated video / demo of this based on the latest PRs you've put in?

Copy link
Contributor

@pgayvallet pgayvallet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like @elastic/response-ops to given their opinion regarding #189027 (comment), as I'm not the one that can decide on that one.

That aside, LGTM

Copy link
Member

@pmuellr pmuellr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments after starting a more thorough review than before :-)

});

export const SecretsSchema = schema.object({
providerSecrets: schema.object({}, { unknowns: 'allow', defaultValue: {} }),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe we've had anyone use unknowns: 'allow' before - and I'm wondering if this will impact updates. I'm not sure this is still the case, but we used to do "partial updates" of some of these things, which is why we don't allow fields to be undefined (null is fine though). Trying to partially update an object with a field x set to some value, with an object that doesn't have an x property, would leave the x property in the updated object.

I'll see if I can test this myself, but would probably be good to have a function test to make sure an update operation on the rule works, with such a scenario.

... a bit later ...

Tested, and it appears to work in config and secrets. I was wondering if AAD (part of encrypted SO's) would be affected, it also does not seem to have issues - to test that you have to try to execute the connector.

I'll ask the team if anyone knows if we specifically support this now, but I'm thinking over the years we've ended up "fixing" this, and maybe not realized it :-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No response from the team, so I think "we're not sure" :-)

Given my tests yesterday, I think it's ok. However, I think you should add a function test where you run the connector once, then update the config and secrets, make sure you can GET the connector back after the update, and then try to execute the connector - it doesn't have to successfully run at the service level, we're wanting to make sure that the encrypted saved object (ESO) doesn't become corrupt because of the update - it would fail during the execute in the alerting framework code to decrypt the ESO, before it even tries to run the connector.

Copy link
Contributor

@jeramysoucy jeramysoucy Oct 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like @pmuellr's suggestion of adding a test to confirm the connector can execute after this type of update.

I believe this works because the SO update operation pulls the ESO document prior to updating the attributes, and is able to effectively perform a full update rather than a partial. And though this may work, I think it is best practice to keep a strict schema for ESOs - particularly for AAD attributes. If the shape of an attribute is not definable with a strict schema, it may not be a good candidate for AAD.

I just want to loop in @azasypkin here as well, as he may have some insight or opinion to share.

Copy link
Member

@azasypkin azasypkin Oct 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the shape of an attribute is not definable with a strict schema, it may not be a good candidate for AAD.

Yes, the decision to include anything in the AAD should always be explicit, and the developer should understand why making a specific attribute part of the AAD makes sense and how this attribute relates to the encrypted value. Otherwise, including an attribute in the AAD makes it harder to reason about the ESO and could also affect the performance and maintainability of operations with such an object. Other than that, I agree with everything said above and +100 to the functional test — it will definitely pay off in the long term.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some context. The object fields being added to this connector's config and secrets end up in those fields in the ESO - secrets is encrypted, config is AAD:

encryptedSavedObjects.registerType({
type: ACTION_SAVED_OBJECT_TYPE,
attributesToEncrypt: new Set(['secrets']),
attributesToIncludeInAAD: new Set(['actionTypeId', 'isMissingSecrets', 'config']),
});

The alerting framework doesn't define the contents of these fields, they are populated by the connector type. To make things easier (on everyone) we do not allow connector developers to specify WHAT should be in AAD, we take their entire config object and treat it as AAD.

If for some reason this is unacceptable, I think the other option is to save these as JSON-ified string values instead, which the connector would then have to serialize / parse in the connector's code. And that would also make using with terraform/etc type tooling more cumbersome.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see, thanks for the context. I didn’t realize it was for the consumers of the alerting framework or second-order consumers of the ESO API. I agree, exposing the notion of AAD to these consumers or adding some additional machinery would make things more complex, confusing, and error-prone. In this case, having a more strictly defined shape for that config on the consumer side, rather than unknown, would at least provide a bit more transparency about what exactly is being used as part of the AAD, even if some of those fields might not be relevant to AAD.

});

export const SparseEmbeddingResponseSchema = schema.object({
sparse_embedding: schema.arrayOf(schema.object({}, { unknowns: 'allow' }), { defaultValue: [] }),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unknowns: 'allow' on params / responses (vs config or secrets) should be fine - given my comment above about usage in config and secrets.

Comment on lines 69 to 79
preSaveHook: async ({ config, secrets, logger, services, isUpdate }) => {
const esClient = services.scopedClusterClient.asCurrentUser;
try {
const taskSettings = config?.taskTypeConfig
? {
...unflattenObject(config?.taskTypeConfig),
}
: {};
const serviceSettings = {
...unflattenObject(config?.providerConfig ?? {}),
...unflattenObject(secrets?.providerSecrets ?? {}),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya, what Yuliia said is correct - I have no idea if not having pre-defined connectors will be a problem for someone.

I think the problem from a response-ops framework side is that this is apparently a connector type that you should not pre-define, and we don’t (currently) have a way of preventing that.

I guess it depends on how obvious the connector is broken in the UX, or responds to execution requests - we'd probably want to indicate in error notifications that these can't be pre-defined. Could also log something at startup if they are found.

@YulNaumenko
Copy link
Contributor Author

Ya, what Yuliia said is correct - I have no idea if not having pre-defined connectors will be a problem for someone.
I think the problem from a response-ops framework side is that this is apparently a connector type that you should not pre-define, and we don’t (currently) have a way of preventing that.
I guess it depends on how obvious the connector is broken in the UX, or responds to execution requests - we'd probably want to indicate in error notifications that these can't be pre-defined. Could also log something at startup if they are found.

@pmuellr Actually we need to be able to to create pre-configured connectors for Inference connector, at least the one for Default Elastic LLM 😀
I was thinking to extend the documentation, with manual handle of Inference Endpoint lifecycle in case of pre-configured connector

@pmuellr
Copy link
Member

pmuellr commented Oct 10, 2024

@pmuellr Actually we need to be able to to create pre-configured connectors for Inference connector, at least the one for Default Elastic LLM 😀 I was thinking to extend the documentation, with manual handle of Inference Endpoint lifecycle in case of pre-configured connector

We could add some plugin start up code that looks for these and creates the inference, but you'd likely want this in a task so only one Kibana will run in. Or maybe the interference endpoint is idempotent and you can just call it multiple times with the same arguments?

Not clear how the inference delete would happen though.

Curious if this is something that WE provide to the customer, or something an on prem customer can do themselves. If it's something WE provide, in cloud, then the params / secrets will need to be baked in, like the email proxy is, in some cloud configuration. And it seems like some cloud startup code could also create the inference? Maybe that's too far removed from Kibana though.

Note that customers can not create preconfigured connectors in any of our hosted cloud envs (ECH, serverless), but can "on prem" offerings.

Copy link
Contributor

@szabosteve szabosteve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some smaller suggestions – all committable –, please take or leave them.

docs/management/connectors/action-types/inference.asciidoc Outdated Show resolved Hide resolved

[role="screenshot"]
image::management/connectors/images/inference-connector.png[{inference} connector]
// NOTE: This is an autogenerated screenshot. Do not edit it directly.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it really auto-generated or is this string a copy/paste error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy/paste

docs/management/connectors/action-types/inference.asciidoc Outdated Show resolved Hide resolved
docs/management/connectors/action-types/inference.asciidoc Outdated Show resolved Hide resolved
docs/management/connectors/action-types/inference.asciidoc Outdated Show resolved Hide resolved
docs/management/connectors/action-types/inference.asciidoc Outdated Show resolved Hide resolved
docs/management/connectors/action-types/inference.asciidoc Outdated Show resolved Hide resolved
docs/management/connectors/action-types/inference.asciidoc Outdated Show resolved Hide resolved
docs/management/connectors/action-types/inference.asciidoc Outdated Show resolved Hide resolved
@adcoelho
Copy link
Contributor

I don't think the fields are being cleared properly when changing between the different services in the form.

Screen.Recording.2024-10-11.at.12.05.00.mov

If I edit the field Model for Amazon Bedrock and then change the service to Mistral the field will be populated with the same value. Maybe that is intended but if I try to save the form I'll get a Field is required. error even though there is text there.

(This happens not just with text but with all fields with the same name across services. Model ID, API Key, etc)

Copy link
Contributor

@adcoelho adcoelho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code-wise approved but I left some comments with bugs I found and with suggested changes to tests.

import { SUB_ACTION } from '../../../common/inference/constants';
import { InferenceActionConnector, InferenceActionParams } from './types';

const InferenceServiceParamsFields: React.FunctionComponent<
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to test these but couldn't find them in Kibana 😅

Normally I'd create a rule and go to the actions but in this case I couldn't find the connector. How does this work?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is like the other gen-ai connectors, it can't be used as an alerting rule action. In theory you should be able to run it via the connector / test UX, but I think someone reported earlier this wasn't quite working.

The product uses this by interacting with the connector via the actionsClient directly. The actions plugin is used since it handles a lot of the issues that are similar to other connectors - common way to "configure" the service, all kinds of validation, checks within our axios usage for TLS/SSL / proxy / etc kind of configuration of the outgoing http connections, etc. It just can't be used as an alerting rule action.

@mdefazio mdefazio self-requested a review October 11, 2024 12:09
Copy link
Contributor

@mdefazio mdefazio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last edit that would be nice to have in:
When setting up a new elasticsearch model (ELSER or E5), is it possible to pre-populate the threads and allocations with 1 / 2 respectively so there's something in there for the user?

Not necessarily a blocker, but would be nice to have.

Update

When editing the connector, the threads/allocation fields are still available, but do not trigger the option to save. I think the expectation would be that these are either disabled / read-only, or would trigger a change. Is it correct that threads is updatable but allocations are not? (or do I have that switched)?

CleanShot 2024-10-11 at 09 26 51

YulNaumenko and others added 3 commits October 11, 2024 07:23
Co-authored-by: István Zoltán Szabó <[email protected]>
…umenko/kibana into kibana-generic-inference-connector
Copy link
Member

@pmuellr pmuellr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

server-side Response Ops code looks good to me, looks like ux code was already reviewed by Response Ops

The comments here are mostly nits. I think we are REALLY going to want a function test where we update the config and secrets properties spec'd as objects with any allowed keys. We want to create the connector, update those properties such that we remove a property already there, and add one that's not there, to both config and secrets. Then do a GET and make sure we can still read the connector, and that it has the expected values. Then execute the connector - we want to make sure the secrets can still be decrypted - the connector will fail early if that's the case, and won't even call the connectors executor function.


beforeEach(() => jest.resetAllMocks());

describe.skip('getTaskTypes', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the .skip should be removed? Or is the test not actually passing yet due to some other work not complete (on the ES side, maybe)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, when we will have a real API

setFieldValue: (fieldName: string, value: unknown) => void
) => {
const taskTypeSuffix = config.taskType ? `${config.taskType}-` : '';
const inferenceEndpointId = `${config.provider}-${taskTypeSuffix}${Math.random()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure you want to use Math.random() here, which is likely not "securely" random?

My first guess is you should create a v4 uuid, which is what we do in alerting for cases like this.

Copy link
Contributor Author

@YulNaumenko YulNaumenko Oct 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did v4 uuid, but got feedback that it's too long and not user friendly, so was asked to use up to 8 symbols. It's not too strict regarding possible dups errors.


export type FieldsConfiguration = Record<string, ConfigProperties>;

export interface Config {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pattern I've seen folks do to access types from schema in public, is to keep the schema and it's types in server, and then import type those in the public code, right from server. You can only do import type like that, import by itself won't work (well, it will cause problems, but I think it does kinda work).

That will ensure there's a single definition for these types.

);
} catch (e) {
logger.warn(
`Failed to delete inference endpoint for task type "${taskType}" and inference id ${inferenceId}. Error: ${e.message}`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our experience has been that there is never enough information written out for these errors :-)

It would be nice to log the status code at least, not clear if there might be other interesting bits in the error that would be useful to log.

});
inferenceExists = true;
} catch (e) {
/* throws error if inference endpoint by id does not exist */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are other reasons why an error could be thrown, even if it does exist. Probably would be good to specifically check if this is a "not found" error (we do this somewhere in alerting, we can look up the magic incantation) vs something else, and if it's something else, throw THAT error.

export class InferenceSimulator extends Simulator {
private readonly returnError: boolean;

constructor({ returnError = false, proxy }: { returnError?: boolean; proxy?: ProxyArgs }) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some of our simulators, we create a single simulator, and determine how to react based on the request coming in. Maybe a field is set to returnError or such, then it returns error instead of the response. May not work for this case, but thought I'd mention it.

simulator.close();
});

it('should return 200 when creating the connector without a default model', async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests should be also checking if the interference did or did not get created, right? And also testing some error conditions, I guess like the case where the same infererence would be created, and so the pre-save hook fails?

Seems like we will also want some delete tests, checking to make sure the inference gets deleted. Maybe one where the inference is ALREADY deleted, but not the connector. What happens then?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened a follow up ticket to add better tests coverage for this PR #196032

@YulNaumenko YulNaumenko enabled auto-merge (squash) October 13, 2024 19:01
@YulNaumenko YulNaumenko merged commit 288d41d into elastic:main Oct 13, 2024
38 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11317544203

@elasticmachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #6 / Alerting builtin alertTypes es_query rule runs correctly: threshold on ungrouped hit count < > for searchSource search type

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
stackConnectors 275 305 +30

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
integrationAssistant 959.8KB 959.9KB +132.0B
securitySolution 20.8MB 20.8MB +264.0B
stackConnectors 584.1KB 681.3KB +97.3KB
triggersActionsUi 1.6MB 1.6MB +106.0B
total +97.7KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
actions 17.1KB 17.1KB -11.0B
stackConnectors 56.1KB 58.3KB +2.2KB
total +2.2KB
Unknown metric groups

async chunk count

id before after diff
stackConnectors 92 95 +3

ESLint disabled line counts

id before after diff
@kbn/test-suites-xpack 726 727 +1
stackConnectors 119 129 +10
total +11

miscellaneous assets size

id before after diff
stackConnectors 0.0B 34.8KB +34.8KB

Total ESLint disabled count

id before after diff
@kbn/test-suites-xpack 751 752 +1
stackConnectors 125 135 +10
total +11

History

cc @YulNaumenko

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 13, 2024
## Summary
Resolves elastic#188043

This PR adds new connector which is define integration with Elastic
Inference Endpoint via [Inference
APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html)
The lifecycle of the Inference Endpoint are managed by the connector
registered handlers:

- `preSaveHook` -
[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)
new Inference Endpoint in the connector create mode (`isEdit === false`)
and
[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)+[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)
in the connector edit mode (`isEdit === true`)
- `postSaveHook` - check if the connector SO was created/updated and if
not removes Inference Endpoint from preSaveHook
- `postDeleteHook` -
[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)
Inference Endpoint if connector was deleted.

In the Kibana Stack Management Connectors, its represented with the new
card (Technical preview badge):

<img width="1261" alt="Screenshot 2024-09-27 at 2 11 12 PM"
src="https://github.com/user-attachments/assets/dcbcce1f-06e7-4d08-8b77-0ba4105354f8">

To simplify the future integration with AI Assistants, the Connector
consists from the two main UI parts: provider selector and required
provider settings, which will be always displayed
<img width="862" alt="Screenshot 2024-10-07 at 7 59 09 AM"
src="https://github.com/user-attachments/assets/87bae493-c642-479e-b28f-6150354608dd">

and Additional options, which contains optional provider settings and
Task Type configuration:

<img width="861" alt="Screenshot 2024-10-07 at 8 00 15 AM"
src="https://github.com/user-attachments/assets/2341c034-6198-4731-8ce7-e22e6c6fb20f">

subActions corresponds to the different taskTypes Inference API
supports. Each of the task type has its own Inference Perform params.
Currently added:

- completion & completionStream
- rerank
- text_embedding
- sparse_embedding

Follow up work:

1. Collapse/expand Additional options, when the connector flyout/modal
has AI Assistant as a context (path through the extending context
implementation on the connector framework level)
2. Add support for additional params for Completion subAction to be able
to path functions
3. Add support for tokens usage Dashboard, when inference API will
include the used tokens count in the response
4. Add functionality and UX for migration from existing specific AI
connectors to the Inference connector with proper provider and
completion task
5. Integrate Connector with the AI Assistants

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: István Zoltán Szabó <[email protected]>
Co-authored-by: Liam Thompson <[email protected]>
Co-authored-by: Steph Milovic <[email protected]>
(cherry picked from commit 288d41d)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 13, 2024
#196035)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Connectors][GenAI] Inference Service Kibana connector
(#189027)](#189027)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Yuliia
Naumenko","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-13T20:39:09Z","message":"[Connectors][GenAI]
Inference Service Kibana connector (#189027)\n\n## Summary\r\nResolves
https://github.com/elastic/kibana/issues/188043\r\n\r\nThis PR adds new
connector which is define integration with Elastic\r\nInference Endpoint
via
[Inference\r\nAPIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html)\r\nThe
lifecycle of the Inference Endpoint are managed by the
connector\r\nregistered handlers:\r\n\r\n- `preSaveHook`
-\r\n[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nnew
Inference Endpoint in the connector create mode (`isEdit ===
false`)\r\nand\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)+[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nin
the connector edit mode (`isEdit === true`)\r\n- `postSaveHook` - check
if the connector SO was created/updated and if\r\nnot removes Inference
Endpoint from preSaveHook\r\n- `postDeleteHook`
-\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)\r\nInference
Endpoint if connector was deleted.\r\n\r\nIn the Kibana Stack Management
Connectors, its represented with the new\r\ncard (Technical preview
badge):\r\n\r\n<img width=\"1261\" alt=\"Screenshot 2024-09-27 at 2 11
12 PM\"\r\nsrc=\"https://github.com/user-attachments/assets/dcbcce1f-06e7-4d08-8b77-0ba4105354f8\">\r\n\r\nTo
simplify the future integration with AI Assistants, the
Connector\r\nconsists from the two main UI parts: provider selector and
required\r\nprovider settings, which will be always displayed\r\n<img
width=\"862\" alt=\"Screenshot 2024-10-07 at 7 59
09 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/87bae493-c642-479e-b28f-6150354608dd\">\r\n\r\nand
Additional options, which contains optional provider settings
and\r\nTask Type configuration:\r\n\r\n<img width=\"861\"
alt=\"Screenshot 2024-10-07 at 8 00
15 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/2341c034-6198-4731-8ce7-e22e6c6fb20f\">\r\n\r\n\r\nsubActions
corresponds to the different taskTypes Inference API\r\nsupports. Each
of the task type has its own Inference Perform params.\r\nCurrently
added:\r\n\r\n- completion & completionStream\r\n- rerank\r\n-
text_embedding\r\n- sparse_embedding\r\n\r\nFollow up work:\r\n\r\n1.
Collapse/expand Additional options, when the connector
flyout/modal\r\nhas AI Assistant as a context (path through the
extending context\r\nimplementation on the connector framework
level)\r\n2. Add support for additional params for Completion subAction
to be able\r\nto path functions\r\n3. Add support for tokens usage
Dashboard, when inference API will\r\ninclude the used tokens count in
the response\r\n4. Add functionality and UX for migration from existing
specific AI\r\nconnectors to the Inference connector with proper
provider and\r\ncompletion task\r\n5. Integrate Connector with the AI
Assistants\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>\r\nCo-authored-by:
István Zoltán Szabó <[email protected]>\r\nCo-authored-by: Liam
Thompson
<[email protected]>\r\nCo-authored-by: Steph
Milovic
<[email protected]>","sha":"288d41d61ec2389b2e8856da75fd0f3107f9c484","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["v9.0.0","release_note:feature","Feature:Actions/ConnectorTypes","8.16
candidate","v8.16.0","backport:version"],"title":"[Connectors][GenAI]
Inference Service Kibana
connector","number":189027,"url":"https://github.com/elastic/kibana/pull/189027","mergeCommit":{"message":"[Connectors][GenAI]
Inference Service Kibana connector (#189027)\n\n## Summary\r\nResolves
https://github.com/elastic/kibana/issues/188043\r\n\r\nThis PR adds new
connector which is define integration with Elastic\r\nInference Endpoint
via
[Inference\r\nAPIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html)\r\nThe
lifecycle of the Inference Endpoint are managed by the
connector\r\nregistered handlers:\r\n\r\n- `preSaveHook`
-\r\n[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nnew
Inference Endpoint in the connector create mode (`isEdit ===
false`)\r\nand\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)+[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nin
the connector edit mode (`isEdit === true`)\r\n- `postSaveHook` - check
if the connector SO was created/updated and if\r\nnot removes Inference
Endpoint from preSaveHook\r\n- `postDeleteHook`
-\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)\r\nInference
Endpoint if connector was deleted.\r\n\r\nIn the Kibana Stack Management
Connectors, its represented with the new\r\ncard (Technical preview
badge):\r\n\r\n<img width=\"1261\" alt=\"Screenshot 2024-09-27 at 2 11
12 PM\"\r\nsrc=\"https://github.com/user-attachments/assets/dcbcce1f-06e7-4d08-8b77-0ba4105354f8\">\r\n\r\nTo
simplify the future integration with AI Assistants, the
Connector\r\nconsists from the two main UI parts: provider selector and
required\r\nprovider settings, which will be always displayed\r\n<img
width=\"862\" alt=\"Screenshot 2024-10-07 at 7 59
09 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/87bae493-c642-479e-b28f-6150354608dd\">\r\n\r\nand
Additional options, which contains optional provider settings
and\r\nTask Type configuration:\r\n\r\n<img width=\"861\"
alt=\"Screenshot 2024-10-07 at 8 00
15 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/2341c034-6198-4731-8ce7-e22e6c6fb20f\">\r\n\r\n\r\nsubActions
corresponds to the different taskTypes Inference API\r\nsupports. Each
of the task type has its own Inference Perform params.\r\nCurrently
added:\r\n\r\n- completion & completionStream\r\n- rerank\r\n-
text_embedding\r\n- sparse_embedding\r\n\r\nFollow up work:\r\n\r\n1.
Collapse/expand Additional options, when the connector
flyout/modal\r\nhas AI Assistant as a context (path through the
extending context\r\nimplementation on the connector framework
level)\r\n2. Add support for additional params for Completion subAction
to be able\r\nto path functions\r\n3. Add support for tokens usage
Dashboard, when inference API will\r\ninclude the used tokens count in
the response\r\n4. Add functionality and UX for migration from existing
specific AI\r\nconnectors to the Inference connector with proper
provider and\r\ncompletion task\r\n5. Integrate Connector with the AI
Assistants\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>\r\nCo-authored-by:
István Zoltán Szabó <[email protected]>\r\nCo-authored-by: Liam
Thompson
<[email protected]>\r\nCo-authored-by: Steph
Milovic
<[email protected]>","sha":"288d41d61ec2389b2e8856da75fd0f3107f9c484"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/189027","number":189027,"mergeCommit":{"message":"[Connectors][GenAI]
Inference Service Kibana connector (#189027)\n\n## Summary\r\nResolves
https://github.com/elastic/kibana/issues/188043\r\n\r\nThis PR adds new
connector which is define integration with Elastic\r\nInference Endpoint
via
[Inference\r\nAPIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html)\r\nThe
lifecycle of the Inference Endpoint are managed by the
connector\r\nregistered handlers:\r\n\r\n- `preSaveHook`
-\r\n[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nnew
Inference Endpoint in the connector create mode (`isEdit ===
false`)\r\nand\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)+[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nin
the connector edit mode (`isEdit === true`)\r\n- `postSaveHook` - check
if the connector SO was created/updated and if\r\nnot removes Inference
Endpoint from preSaveHook\r\n- `postDeleteHook`
-\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)\r\nInference
Endpoint if connector was deleted.\r\n\r\nIn the Kibana Stack Management
Connectors, its represented with the new\r\ncard (Technical preview
badge):\r\n\r\n<img width=\"1261\" alt=\"Screenshot 2024-09-27 at 2 11
12 PM\"\r\nsrc=\"https://github.com/user-attachments/assets/dcbcce1f-06e7-4d08-8b77-0ba4105354f8\">\r\n\r\nTo
simplify the future integration with AI Assistants, the
Connector\r\nconsists from the two main UI parts: provider selector and
required\r\nprovider settings, which will be always displayed\r\n<img
width=\"862\" alt=\"Screenshot 2024-10-07 at 7 59
09 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/87bae493-c642-479e-b28f-6150354608dd\">\r\n\r\nand
Additional options, which contains optional provider settings
and\r\nTask Type configuration:\r\n\r\n<img width=\"861\"
alt=\"Screenshot 2024-10-07 at 8 00
15 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/2341c034-6198-4731-8ce7-e22e6c6fb20f\">\r\n\r\n\r\nsubActions
corresponds to the different taskTypes Inference API\r\nsupports. Each
of the task type has its own Inference Perform params.\r\nCurrently
added:\r\n\r\n- completion & completionStream\r\n- rerank\r\n-
text_embedding\r\n- sparse_embedding\r\n\r\nFollow up work:\r\n\r\n1.
Collapse/expand Additional options, when the connector
flyout/modal\r\nhas AI Assistant as a context (path through the
extending context\r\nimplementation on the connector framework
level)\r\n2. Add support for additional params for Completion subAction
to be able\r\nto path functions\r\n3. Add support for tokens usage
Dashboard, when inference API will\r\ninclude the used tokens count in
the response\r\n4. Add functionality and UX for migration from existing
specific AI\r\nconnectors to the Inference connector with proper
provider and\r\ncompletion task\r\n5. Integrate Connector with the AI
Assistants\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>\r\nCo-authored-by:
István Zoltán Szabó <[email protected]>\r\nCo-authored-by: Liam
Thompson
<[email protected]>\r\nCo-authored-by: Steph
Milovic
<[email protected]>","sha":"288d41d61ec2389b2e8856da75fd0f3107f9c484"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Yuliia Naumenko <[email protected]>
kilfoyle added a commit to kilfoyle/kibana that referenced this pull request Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.16 candidate backport:version Backport to applied version labels Feature:Actions/ConnectorTypes Issues related to specific Connector Types on the Actions Framework release_note:feature Makes this part of the condensed release notes v8.16.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Connectors] Generic Kibana connector for Inference API