-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Connectors][GenAI] Inference Service Kibana connector #189027
[Connectors][GenAI] Inference Service Kibana connector #189027
Conversation
A documentation preview will be available soon. Request a new doc build by commenting
If your PR continues to fail for an unknown reason, the doc build pipeline may be broken. Elastic employees can check the pipeline status here. |
This comment was marked as resolved.
This comment was marked as resolved.
@serenachou is there a link to said video demo? |
@daveyholler it's in the slack thread I added you and Julian to : https://elastic.slack.com/archives/C07DCCPGB55/p1721799374980719 |
For simplicity, once Serena's suggestions have been drafted, please re-ping and we can copyedit from there :) |
This comment was marked as resolved.
This comment was marked as resolved.
I cannot change it, because it's a part of the connectors framework and UX consistency. Buttons are also inherited from the connectors framework and there is no way to manage it's styles. |
This comment was marked as resolved.
This comment was marked as resolved.
…o being able to manage inference endpoint lifecicle with hooks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few copyedits per @serenachou's drafts
x-pack/plugins/stack_connectors/public/connector_types/inference/connector.tsx
Outdated
Show resolved
Hide resolved
x-pack/plugins/stack_connectors/public/connector_types/inference/connector.tsx
Outdated
Show resolved
Hide resolved
x-pack/plugins/stack_connectors/public/connector_types/inference/inference.tsx
Outdated
Show resolved
Hide resolved
…rence-connector # Conflicts: # docs/settings/alert-action-settings.asciidoc # oas_docs/output/kibana.serverless.yaml # oas_docs/output/kibana.yaml # x-pack/plugins/actions/docs/openapi/bundled.yaml
…rence-connector # Conflicts: # oas_docs/output/kibana.serverless.yaml # oas_docs/output/kibana.yaml
@YulNaumenko will you have an updated video / demo of this based on the latest PRs you've put in? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like @elastic/response-ops to given their opinion regarding #189027 (comment), as I'm not the one that can decide on that one.
That aside, LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some comments after starting a more thorough review than before :-)
}); | ||
|
||
export const SecretsSchema = schema.object({ | ||
providerSecrets: schema.object({}, { unknowns: 'allow', defaultValue: {} }), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't believe we've had anyone use unknowns: 'allow'
before - and I'm wondering if this will impact updates. I'm not sure this is still the case, but we used to do "partial updates" of some of these things, which is why we don't allow fields to be undefined
(null
is fine though). Trying to partially update an object with a field x
set to some value, with an object that doesn't have an x
property, would leave the x
property in the updated object.
I'll see if I can test this myself, but would probably be good to have a function test to make sure an update
operation on the rule works, with such a scenario.
... a bit later ...
Tested, and it appears to work in config
and secrets
. I was wondering if AAD (part of encrypted SO's) would be affected, it also does not seem to have issues - to test that you have to try to execute the connector.
I'll ask the team if anyone knows if we specifically support this now, but I'm thinking over the years we've ended up "fixing" this, and maybe not realized it :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No response from the team, so I think "we're not sure" :-)
Given my tests yesterday, I think it's ok. However, I think you should add a function test where you run the connector once, then update the config and secrets, make sure you can GET the connector back after the update, and then try to execute the connector - it doesn't have to successfully run at the service level, we're wanting to make sure that the encrypted saved object (ESO) doesn't become corrupt because of the update - it would fail during the execute in the alerting framework code to decrypt the ESO, before it even tries to run the connector.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like @pmuellr's suggestion of adding a test to confirm the connector can execute after this type of update.
I believe this works because the SO update operation pulls the ESO document prior to updating the attributes, and is able to effectively perform a full update rather than a partial. And though this may work, I think it is best practice to keep a strict schema for ESOs - particularly for AAD attributes. If the shape of an attribute is not definable with a strict schema, it may not be a good candidate for AAD.
I just want to loop in @azasypkin here as well, as he may have some insight or opinion to share.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the shape of an attribute is not definable with a strict schema, it may not be a good candidate for AAD.
Yes, the decision to include anything in the AAD should always be explicit, and the developer should understand why making a specific attribute part of the AAD makes sense and how this attribute relates to the encrypted value. Otherwise, including an attribute in the AAD makes it harder to reason about the ESO and could also affect the performance and maintainability of operations with such an object. Other than that, I agree with everything said above and +100 to the functional test — it will definitely pay off in the long term.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some context. The object fields being added to this connector's config and secrets end up in those fields in the ESO - secrets
is encrypted, config
is AAD:
kibana/x-pack/plugins/actions/server/saved_objects/index.ts
Lines 72 to 76 in 920d782
encryptedSavedObjects.registerType({ | |
type: ACTION_SAVED_OBJECT_TYPE, | |
attributesToEncrypt: new Set(['secrets']), | |
attributesToIncludeInAAD: new Set(['actionTypeId', 'isMissingSecrets', 'config']), | |
}); |
The alerting framework doesn't define the contents of these fields, they are populated by the connector type. To make things easier (on everyone) we do not allow connector developers to specify WHAT should be in AAD, we take their entire config object and treat it as AAD.
If for some reason this is unacceptable, I think the other option is to save these as JSON-ified string values instead, which the connector would then have to serialize / parse in the connector's code. And that would also make using with terraform/etc type tooling more cumbersome.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I see, thanks for the context. I didn’t realize it was for the consumers of the alerting framework or second-order consumers of the ESO API. I agree, exposing the notion of AAD to these consumers or adding some additional machinery would make things more complex, confusing, and error-prone. In this case, having a more strictly defined shape for that config on the consumer side, rather than unknown, would at least provide a bit more transparency about what exactly is being used as part of the AAD, even if some of those fields might not be relevant to AAD.
}); | ||
|
||
export const SparseEmbeddingResponseSchema = schema.object({ | ||
sparse_embedding: schema.arrayOf(schema.object({}, { unknowns: 'allow' }), { defaultValue: [] }), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The unknowns: 'allow'
on params / responses (vs config
or secrets
) should be fine - given my comment above about usage in config
and secrets
.
preSaveHook: async ({ config, secrets, logger, services, isUpdate }) => { | ||
const esClient = services.scopedClusterClient.asCurrentUser; | ||
try { | ||
const taskSettings = config?.taskTypeConfig | ||
? { | ||
...unflattenObject(config?.taskTypeConfig), | ||
} | ||
: {}; | ||
const serviceSettings = { | ||
...unflattenObject(config?.providerConfig ?? {}), | ||
...unflattenObject(secrets?.providerSecrets ?? {}), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ya, what Yuliia said is correct - I have no idea if not having pre-defined connectors will be a problem for someone.
I think the problem from a response-ops framework side is that this is apparently a connector type that you should not pre-define, and we don’t (currently) have a way of preventing that.
I guess it depends on how obvious the connector is broken in the UX, or responds to execution requests - we'd probably want to indicate in error notifications that these can't be pre-defined. Could also log something at startup if they are found.
Co-authored-by: Steph Milovic <[email protected]>
@pmuellr Actually we need to be able to to create pre-configured connectors for Inference connector, at least the one for Default Elastic LLM 😀 |
We could add some plugin start up code that looks for these and creates the inference, but you'd likely want this in a task so only one Kibana will run in. Or maybe the interference endpoint is idempotent and you can just call it multiple times with the same arguments? Not clear how the inference delete would happen though. Curious if this is something that WE provide to the customer, or something an on prem customer can do themselves. If it's something WE provide, in cloud, then the params / secrets will need to be baked in, like the email proxy is, in some cloud configuration. And it seems like some cloud startup code could also create the inference? Maybe that's too far removed from Kibana though. Note that customers can not create preconfigured connectors in any of our hosted cloud envs (ECH, serverless), but can "on prem" offerings. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left some smaller suggestions – all committable –, please take or leave them.
|
||
[role="screenshot"] | ||
image::management/connectors/images/inference-connector.png[{inference} connector] | ||
// NOTE: This is an autogenerated screenshot. Do not edit it directly. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it really auto-generated or is this string a copy/paste error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
copy/paste
I don't think the fields are being cleared properly when changing between the different services in the form. Screen.Recording.2024-10-11.at.12.05.00.movIf I edit the field (This happens not just with text but with all fields with the same name across services. Model ID, API Key, etc) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code-wise approved but I left some comments with bugs I found and with suggested changes to tests.
x-pack/plugins/stack_connectors/public/connector_types/inference/params.tsx
Outdated
Show resolved
Hide resolved
x-pack/plugins/stack_connectors/public/connector_types/inference/params.tsx
Outdated
Show resolved
Hide resolved
x-pack/plugins/stack_connectors/public/connector_types/inference/inference.test.tsx
Outdated
Show resolved
Hide resolved
x-pack/plugins/stack_connectors/public/connector_types/inference/inference.test.tsx
Show resolved
Hide resolved
x-pack/plugins/stack_connectors/public/connector_types/inference/inference.test.tsx
Show resolved
Hide resolved
import { SUB_ACTION } from '../../../common/inference/constants'; | ||
import { InferenceActionConnector, InferenceActionParams } from './types'; | ||
|
||
const InferenceServiceParamsFields: React.FunctionComponent< |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wanted to test these but couldn't find them in Kibana 😅
Normally I'd create a rule and go to the actions but in this case I couldn't find the connector. How does this work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this is like the other gen-ai connectors, it can't be used as an alerting rule action. In theory you should be able to run it via the connector / test UX, but I think someone reported earlier this wasn't quite working.
The product uses this by interacting with the connector via the actionsClient directly. The actions plugin is used since it handles a lot of the issues that are similar to other connectors - common way to "configure" the service, all kinds of validation, checks within our axios usage for TLS/SSL / proxy / etc kind of configuration of the outgoing http connections, etc. It just can't be used as an alerting rule action.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One last edit that would be nice to have in:
When setting up a new elasticsearch model (ELSER or E5), is it possible to pre-populate the threads and allocations with 1 / 2 respectively so there's something in there for the user?
Not necessarily a blocker, but would be nice to have.
Update
When editing the connector, the threads/allocation fields are still available, but do not trigger the option to save. I think the expectation would be that these are either disabled / read-only, or would trigger a change. Is it correct that threads is updatable but allocations are not? (or do I have that switched)?
Co-authored-by: István Zoltán Szabó <[email protected]>
…umenko/kibana into kibana-generic-inference-connector
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
server-side Response Ops code looks good to me, looks like ux code was already reviewed by Response Ops
The comments here are mostly nits. I think we are REALLY going to want a function test where we update the config and secrets properties spec'd as objects with any allowed keys. We want to create the connector, update those properties such that we remove a property already there, and add one that's not there, to both config and secrets. Then do a GET and make sure we can still read the connector, and that it has the expected values. Then execute the connector - we want to make sure the secrets can still be decrypted - the connector will fail early if that's the case, and won't even call the connectors executor function.
|
||
beforeEach(() => jest.resetAllMocks()); | ||
|
||
describe.skip('getTaskTypes', () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume the .skip
should be removed? Or is the test not actually passing yet due to some other work not complete (on the ES side, maybe)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, when we will have a real API
setFieldValue: (fieldName: string, value: unknown) => void | ||
) => { | ||
const taskTypeSuffix = config.taskType ? `${config.taskType}-` : ''; | ||
const inferenceEndpointId = `${config.provider}-${taskTypeSuffix}${Math.random() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure you want to use Math.random()
here, which is likely not "securely" random?
My first guess is you should create a v4 uuid, which is what we do in alerting for cases like this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did v4 uuid, but got feedback that it's too long and not user friendly, so was asked to use up to 8 symbols. It's not too strict regarding possible dups errors.
|
||
export type FieldsConfiguration = Record<string, ConfigProperties>; | ||
|
||
export interface Config { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The pattern I've seen folks do to access types from schema in public
, is to keep the schema and it's types in server
, and then import type
those in the public
code, right from server
. You can only do import type
like that, import
by itself won't work (well, it will cause problems, but I think it does kinda work).
That will ensure there's a single definition for these types.
x-pack/plugins/stack_connectors/server/connector_types/inference/index.ts
Outdated
Show resolved
Hide resolved
); | ||
} catch (e) { | ||
logger.warn( | ||
`Failed to delete inference endpoint for task type "${taskType}" and inference id ${inferenceId}. Error: ${e.message}` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Our experience has been that there is never enough information written out for these errors :-)
It would be nice to log the status code at least, not clear if there might be other interesting bits in the error that would be useful to log.
}); | ||
inferenceExists = true; | ||
} catch (e) { | ||
/* throws error if inference endpoint by id does not exist */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are other reasons why an error could be thrown, even if it does exist. Probably would be good to specifically check if this is a "not found" error (we do this somewhere in alerting, we can look up the magic incantation) vs something else, and if it's something else, throw THAT error.
x-pack/plugins/stack_connectors/server/connector_types/inference/inference.ts
Outdated
Show resolved
Hide resolved
export class InferenceSimulator extends Simulator { | ||
private readonly returnError: boolean; | ||
|
||
constructor({ returnError = false, proxy }: { returnError?: boolean; proxy?: ProxyArgs }) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For some of our simulators, we create a single simulator, and determine how to react based on the request coming in. Maybe a field is set to returnError
or such, then it returns error instead of the response. May not work for this case, but thought I'd mention it.
simulator.close(); | ||
}); | ||
|
||
it('should return 200 when creating the connector without a default model', async () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These tests should be also checking if the interference did or did not get created, right? And also testing some error conditions, I guess like the case where the same infererence would be created, and so the pre-save hook fails?
Seems like we will also want some delete tests, checking to make sure the inference gets deleted. Maybe one where the inference is ALREADY deleted, but not the connector. What happens then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I opened a follow up ticket to add better tests coverage for this PR #196032
Starting backport for target branches: 8.x |
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Module Count
Async chunks
Page load bundle
Unknown metric groupsasync chunk count
ESLint disabled line counts
miscellaneous assets size
Total ESLint disabled count
History
cc @YulNaumenko |
## Summary Resolves elastic#188043 This PR adds new connector which is define integration with Elastic Inference Endpoint via [Inference APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html) The lifecycle of the Inference Endpoint are managed by the connector registered handlers: - `preSaveHook` - [create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html) new Inference Endpoint in the connector create mode (`isEdit === false`) and [delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)+[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html) in the connector edit mode (`isEdit === true`) - `postSaveHook` - check if the connector SO was created/updated and if not removes Inference Endpoint from preSaveHook - `postDeleteHook` - [delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html) Inference Endpoint if connector was deleted. In the Kibana Stack Management Connectors, its represented with the new card (Technical preview badge): <img width="1261" alt="Screenshot 2024-09-27 at 2 11 12 PM" src="https://github.com/user-attachments/assets/dcbcce1f-06e7-4d08-8b77-0ba4105354f8"> To simplify the future integration with AI Assistants, the Connector consists from the two main UI parts: provider selector and required provider settings, which will be always displayed <img width="862" alt="Screenshot 2024-10-07 at 7 59 09 AM" src="https://github.com/user-attachments/assets/87bae493-c642-479e-b28f-6150354608dd"> and Additional options, which contains optional provider settings and Task Type configuration: <img width="861" alt="Screenshot 2024-10-07 at 8 00 15 AM" src="https://github.com/user-attachments/assets/2341c034-6198-4731-8ce7-e22e6c6fb20f"> subActions corresponds to the different taskTypes Inference API supports. Each of the task type has its own Inference Perform params. Currently added: - completion & completionStream - rerank - text_embedding - sparse_embedding Follow up work: 1. Collapse/expand Additional options, when the connector flyout/modal has AI Assistant as a context (path through the extending context implementation on the connector framework level) 2. Add support for additional params for Completion subAction to be able to path functions 3. Add support for tokens usage Dashboard, when inference API will include the used tokens count in the response 4. Add functionality and UX for migration from existing specific AI connectors to the Inference connector with proper provider and completion task 5. Integrate Connector with the AI Assistants --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: István Zoltán Szabó <[email protected]> Co-authored-by: Liam Thompson <[email protected]> Co-authored-by: Steph Milovic <[email protected]> (cherry picked from commit 288d41d)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
#196035) # Backport This will backport the following commits from `main` to `8.x`: - [[Connectors][GenAI] Inference Service Kibana connector (#189027)](#189027) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Yuliia Naumenko","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-13T20:39:09Z","message":"[Connectors][GenAI] Inference Service Kibana connector (#189027)\n\n## Summary\r\nResolves https://github.com/elastic/kibana/issues/188043\r\n\r\nThis PR adds new connector which is define integration with Elastic\r\nInference Endpoint via [Inference\r\nAPIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html)\r\nThe lifecycle of the Inference Endpoint are managed by the connector\r\nregistered handlers:\r\n\r\n- `preSaveHook` -\r\n[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nnew Inference Endpoint in the connector create mode (`isEdit === false`)\r\nand\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)+[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nin the connector edit mode (`isEdit === true`)\r\n- `postSaveHook` - check if the connector SO was created/updated and if\r\nnot removes Inference Endpoint from preSaveHook\r\n- `postDeleteHook` -\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)\r\nInference Endpoint if connector was deleted.\r\n\r\nIn the Kibana Stack Management Connectors, its represented with the new\r\ncard (Technical preview badge):\r\n\r\n<img width=\"1261\" alt=\"Screenshot 2024-09-27 at 2 11 12 PM\"\r\nsrc=\"https://github.com/user-attachments/assets/dcbcce1f-06e7-4d08-8b77-0ba4105354f8\">\r\n\r\nTo simplify the future integration with AI Assistants, the Connector\r\nconsists from the two main UI parts: provider selector and required\r\nprovider settings, which will be always displayed\r\n<img width=\"862\" alt=\"Screenshot 2024-10-07 at 7 59 09 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/87bae493-c642-479e-b28f-6150354608dd\">\r\n\r\nand Additional options, which contains optional provider settings and\r\nTask Type configuration:\r\n\r\n<img width=\"861\" alt=\"Screenshot 2024-10-07 at 8 00 15 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/2341c034-6198-4731-8ce7-e22e6c6fb20f\">\r\n\r\n\r\nsubActions corresponds to the different taskTypes Inference API\r\nsupports. Each of the task type has its own Inference Perform params.\r\nCurrently added:\r\n\r\n- completion & completionStream\r\n- rerank\r\n- text_embedding\r\n- sparse_embedding\r\n\r\nFollow up work:\r\n\r\n1. Collapse/expand Additional options, when the connector flyout/modal\r\nhas AI Assistant as a context (path through the extending context\r\nimplementation on the connector framework level)\r\n2. Add support for additional params for Completion subAction to be able\r\nto path functions\r\n3. Add support for tokens usage Dashboard, when inference API will\r\ninclude the used tokens count in the response\r\n4. Add functionality and UX for migration from existing specific AI\r\nconnectors to the Inference connector with proper provider and\r\ncompletion task\r\n5. Integrate Connector with the AI Assistants\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>\r\nCo-authored-by: István Zoltán Szabó <[email protected]>\r\nCo-authored-by: Liam Thompson <[email protected]>\r\nCo-authored-by: Steph Milovic <[email protected]>","sha":"288d41d61ec2389b2e8856da75fd0f3107f9c484","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["v9.0.0","release_note:feature","Feature:Actions/ConnectorTypes","8.16 candidate","v8.16.0","backport:version"],"title":"[Connectors][GenAI] Inference Service Kibana connector","number":189027,"url":"https://github.com/elastic/kibana/pull/189027","mergeCommit":{"message":"[Connectors][GenAI] Inference Service Kibana connector (#189027)\n\n## Summary\r\nResolves https://github.com/elastic/kibana/issues/188043\r\n\r\nThis PR adds new connector which is define integration with Elastic\r\nInference Endpoint via [Inference\r\nAPIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html)\r\nThe lifecycle of the Inference Endpoint are managed by the connector\r\nregistered handlers:\r\n\r\n- `preSaveHook` -\r\n[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nnew Inference Endpoint in the connector create mode (`isEdit === false`)\r\nand\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)+[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nin the connector edit mode (`isEdit === true`)\r\n- `postSaveHook` - check if the connector SO was created/updated and if\r\nnot removes Inference Endpoint from preSaveHook\r\n- `postDeleteHook` -\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)\r\nInference Endpoint if connector was deleted.\r\n\r\nIn the Kibana Stack Management Connectors, its represented with the new\r\ncard (Technical preview badge):\r\n\r\n<img width=\"1261\" alt=\"Screenshot 2024-09-27 at 2 11 12 PM\"\r\nsrc=\"https://github.com/user-attachments/assets/dcbcce1f-06e7-4d08-8b77-0ba4105354f8\">\r\n\r\nTo simplify the future integration with AI Assistants, the Connector\r\nconsists from the two main UI parts: provider selector and required\r\nprovider settings, which will be always displayed\r\n<img width=\"862\" alt=\"Screenshot 2024-10-07 at 7 59 09 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/87bae493-c642-479e-b28f-6150354608dd\">\r\n\r\nand Additional options, which contains optional provider settings and\r\nTask Type configuration:\r\n\r\n<img width=\"861\" alt=\"Screenshot 2024-10-07 at 8 00 15 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/2341c034-6198-4731-8ce7-e22e6c6fb20f\">\r\n\r\n\r\nsubActions corresponds to the different taskTypes Inference API\r\nsupports. Each of the task type has its own Inference Perform params.\r\nCurrently added:\r\n\r\n- completion & completionStream\r\n- rerank\r\n- text_embedding\r\n- sparse_embedding\r\n\r\nFollow up work:\r\n\r\n1. Collapse/expand Additional options, when the connector flyout/modal\r\nhas AI Assistant as a context (path through the extending context\r\nimplementation on the connector framework level)\r\n2. Add support for additional params for Completion subAction to be able\r\nto path functions\r\n3. Add support for tokens usage Dashboard, when inference API will\r\ninclude the used tokens count in the response\r\n4. Add functionality and UX for migration from existing specific AI\r\nconnectors to the Inference connector with proper provider and\r\ncompletion task\r\n5. Integrate Connector with the AI Assistants\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>\r\nCo-authored-by: István Zoltán Szabó <[email protected]>\r\nCo-authored-by: Liam Thompson <[email protected]>\r\nCo-authored-by: Steph Milovic <[email protected]>","sha":"288d41d61ec2389b2e8856da75fd0f3107f9c484"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/189027","number":189027,"mergeCommit":{"message":"[Connectors][GenAI] Inference Service Kibana connector (#189027)\n\n## Summary\r\nResolves https://github.com/elastic/kibana/issues/188043\r\n\r\nThis PR adds new connector which is define integration with Elastic\r\nInference Endpoint via [Inference\r\nAPIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html)\r\nThe lifecycle of the Inference Endpoint are managed by the connector\r\nregistered handlers:\r\n\r\n- `preSaveHook` -\r\n[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nnew Inference Endpoint in the connector create mode (`isEdit === false`)\r\nand\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)+[create](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html)\r\nin the connector edit mode (`isEdit === true`)\r\n- `postSaveHook` - check if the connector SO was created/updated and if\r\nnot removes Inference Endpoint from preSaveHook\r\n- `postDeleteHook` -\r\n[delete](https://www.elastic.co/guide/en/elasticsearch/reference/current/delete-inference-api.html)\r\nInference Endpoint if connector was deleted.\r\n\r\nIn the Kibana Stack Management Connectors, its represented with the new\r\ncard (Technical preview badge):\r\n\r\n<img width=\"1261\" alt=\"Screenshot 2024-09-27 at 2 11 12 PM\"\r\nsrc=\"https://github.com/user-attachments/assets/dcbcce1f-06e7-4d08-8b77-0ba4105354f8\">\r\n\r\nTo simplify the future integration with AI Assistants, the Connector\r\nconsists from the two main UI parts: provider selector and required\r\nprovider settings, which will be always displayed\r\n<img width=\"862\" alt=\"Screenshot 2024-10-07 at 7 59 09 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/87bae493-c642-479e-b28f-6150354608dd\">\r\n\r\nand Additional options, which contains optional provider settings and\r\nTask Type configuration:\r\n\r\n<img width=\"861\" alt=\"Screenshot 2024-10-07 at 8 00 15 AM\"\r\nsrc=\"https://github.com/user-attachments/assets/2341c034-6198-4731-8ce7-e22e6c6fb20f\">\r\n\r\n\r\nsubActions corresponds to the different taskTypes Inference API\r\nsupports. Each of the task type has its own Inference Perform params.\r\nCurrently added:\r\n\r\n- completion & completionStream\r\n- rerank\r\n- text_embedding\r\n- sparse_embedding\r\n\r\nFollow up work:\r\n\r\n1. Collapse/expand Additional options, when the connector flyout/modal\r\nhas AI Assistant as a context (path through the extending context\r\nimplementation on the connector framework level)\r\n2. Add support for additional params for Completion subAction to be able\r\nto path functions\r\n3. Add support for tokens usage Dashboard, when inference API will\r\ninclude the used tokens count in the response\r\n4. Add functionality and UX for migration from existing specific AI\r\nconnectors to the Inference connector with proper provider and\r\ncompletion task\r\n5. Integrate Connector with the AI Assistants\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>\r\nCo-authored-by: István Zoltán Szabó <[email protected]>\r\nCo-authored-by: Liam Thompson <[email protected]>\r\nCo-authored-by: Steph Milovic <[email protected]>","sha":"288d41d61ec2389b2e8856da75fd0f3107f9c484"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Yuliia Naumenko <[email protected]>
Summary
Resolves #188043
This PR adds new connector which is define integration with Elastic Inference Endpoint via Inference APIs
The lifecycle of the Inference Endpoint are managed by the connector registered handlers:
preSaveHook
- create new Inference Endpoint in the connector create mode (isEdit === false
) and delete+create in the connector edit mode (isEdit === true
)postSaveHook
- check if the connector SO was created/updated and if not removes Inference Endpoint from preSaveHookpostDeleteHook
- delete Inference Endpoint if connector was deleted.In the Kibana Stack Management Connectors, its represented with the new card (Technical preview badge):
To simplify the future integration with AI Assistants, the Connector consists from the two main UI parts: provider selector and required provider settings, which will be always displayed
and Additional options, which contains optional provider settings and Task Type configuration:
subActions corresponds to the different taskTypes Inference API supports. Each of the task type has its own Inference Perform params. Currently added:
Follow up work: