Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat] Kepler.gl AI Assistant #2735

Open
wants to merge 18 commits into
base: master
Choose a base branch
from
Open

[Feat] Kepler.gl AI Assistant #2735

wants to merge 18 commits into from

Conversation

lixun910
Copy link
Collaborator

@lixun910 lixun910 commented Nov 8, 2024

This is a WIP pr that tries to add AI Assistant which was originally developed for https://geoda.ai and now is open sourced and available to use.

  • AI Assistant side panel UI
  • AI Providers
    • OpenAI
    • Google Gemini
    • Ollama (runs on your computer, no token needed)
  • AI features
    • this pr shows an example of call basemap feature from LLM, prompts like "can you help me to change basemap style?"
    • details should be discussed later @ibgreen @heshan0131
  • Builtin features
    • take screenshot to ask
    • voice to text

keplergl-ai

Copy link

netlify bot commented Nov 8, 2024

Deploy Preview for keplergl2 failed. Why did it fail? →

Name Link
🔨 Latest commit 1a316a6
🔍 Latest deploy log https://app.netlify.com/sites/keplergl2/deploys/673bf8386d99f20008aa78cb

@ibgreen
Copy link
Collaborator

ibgreen commented Nov 8, 2024

@lixun910 Very cool. There are a couple of things that need to be discussed, good if we can connect on slack for the first alignment.

Copy link
Collaborator

@ibgreen ibgreen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very exciting.

My personal take is that it would be good to separate out the AI integration more clearly.
At the extreme end, it could go into its own top-level module (@kepler.gl/ai-assistant), but at minimum, having a clearly separated integration code that is not located inside UI glued code like components, hooks, reducers.

Of course we can refine that step-by-step, so approving for now.

@@ -576,3 +577,13 @@ export type TypedFilter =
| TypedSelectFilter
| TypedMultiSelectFilter
| TypedPolygonFilter;

export type AiAssistantConfig = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I though we had merged all d.ts files into .ts...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: For new types, could we add TSDoc /** */ comments on the type and each field (this integrates into vscode intellisense).

Especially fields like temperature and topP - maybe not instantly clear what they are used for?

@@ -201,7 +201,9 @@ export default {
delete: 'Deletar',
timePlayback: 'Tempo de reprodução',
cloudStorage: 'Armazenamento Cloud',
'3DMap': ' Mapa 3D'
'3DMap': ' Mapa 3D',
showAiAssistantPanel: 'Mostrar Assistente IA',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: Did you use AI to translate the AI settings :)

import {MapControlButton} from '../common/styled-components';
import MapControlTooltipFactory from './map-control-tooltip';

interface AiAssistantControlIcons {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Any reason to use interface instead of type here, you seem to be mixing them.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add TSDoc /** */ on the line before each new type and field? Costs little and integrates with vscode.

import { appInjector } from '../container';
import { MapStyle } from '@kepler.gl/reducers';

export function basemapFunctionDefinition(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this the actual integration? I am not sure we should have that mixes with React and state management... it would be good to have clear separation.

@ilyabo
Copy link
Collaborator

ilyabo commented Nov 14, 2024

woohoo, awesome stuff!

@lixun910
Copy link
Collaborator Author

@igorDykhta Hi Igor, do you have any suggestions on why the netlify build failed: https://app.netlify.com/sites/keplergl/deploys/67390c9f870e2b0008a92487 Thank you!

@igorDykhta
Copy link
Collaborator

igorDykhta commented Nov 16, 2024

@igorDykhta Hi Igor, do you have any suggestions on why the netlify build failed: https://app.netlify.com/sites/keplergl/deploys/67390c9f870e2b0008a92487 Thank you!

Have you tried updating yarn.lock?

@igorDykhta igorDykhta closed this Nov 16, 2024
@igorDykhta igorDykhta reopened this Nov 16, 2024
@lixun910
Copy link
Collaborator Author

@igorDykhta Hi Igor, do you have any suggestions on why the netlify build failed: https://app.netlify.com/sites/keplergl/deploys/67390c9f870e2b0008a92487 Thank you!

Have you tried updating yarn.lock?

Thanks, Igor! Yes, I did. Is this causing the compiling error?

@igorDykhta
Copy link
Collaborator

@igorDykhta Hi Igor, do you have any suggestions on why the netlify build failed: https://app.netlify.com/sites/keplergl/deploys/67390c9f870e2b0008a92487 Thank you!

Have you tried updating yarn.lock?

Thanks, Igor! Yes, I did. Is this causing the compiling error?

Do you have access to the failing Netlify Deploy log?

@lixun910
Copy link
Collaborator Author

@igorDykhta Hi Igor, do you have any suggestions on why the netlify build failed: https://app.netlify.com/sites/keplergl/deploys/67390c9f870e2b0008a92487 Thank you!

Have you tried updating yarn.lock?

I think I figure it out: there are two yarn.lock files need to be updated. Thanks, Igor!

@lixun910
Copy link
Collaborator Author

init work done

  • scafolding
  • two exmaple function calling
    • load data from url
    • change basemap
      see the screen recording
      keplergl-ai-1

@@ -197,7 +200,7 @@
"tape": "^4.9.2",
"tape-catch": "^1.0.6",
"typedoc-plugin-markdown": "^3.0.11",
"typescript": "4.5.5",
"typescript": "4.7.2",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

support latest langchain

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants