feat: add support for modin #907
Merged
+834
−87
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Warning Rate Limit Exceeded@mspronesti has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 28 minutes and 14 seconds before requesting another review. How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. WalkthroughThe Changes
Assessment against linked issues
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Type: Enhancement
PR Summary: The pull request introduces support for the Modin library as an alternative DataFrame engine in the pandasai package. It includes changes to dynamically switch between Pandas and Modin, conditional imports based on the selected engine, and updates to various components to accommodate the new engine option.
Decision: Comment
📝 Type: 'Enhancement' - not supported yet.
- Sourcery currently only approves 'Typo fix' PRs.
✅ Issue addressed: this change correctly addresses the issue or implements the desired feature.
No details provided.
📝 Complexity: the changes are too large or complex for Sourcery to approve.
- Unsupported files: the diff contains files that Sourcery does not currently support during reviews.
General suggestions:
- Ensure that the dynamic engine switching functionality is thoroughly documented, including any limitations or considerations that users should be aware of when changing the engine.
- Review the use of the 'DataFrameType' to ensure it aligns with the intended types that the 'df_type' function can handle, as the inclusion of 'str' may not be necessary.
- Verify that the 'engine' variable is properly defined before use in the '_truncate_head_columns' method to prevent potential 'NameError' exceptions.
- Consider simplifying the conditional check in the '_load_engine' method of the SmartDataframe class to improve code readability and maintainability.
Thanks for using Sourcery. We offer it for free for open source projects and would be very grateful if you could help us grow. If you like it, would you consider sharing Sourcery on your favourite social media? ✨
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review Status
Actionable comments generated: 3
Configuration used: CodeRabbit UI
Files ignored due to path filters (2)
poetry.lockis excluded by:!**/*.lockpyproject.tomlis excluded by:!**/*.toml
Files selected for processing (20)
- pandasai/init.py (2 hunks)
- pandasai/connectors/airtable.py (1 hunks)
- pandasai/connectors/snowflake.py (1 hunks)
- pandasai/connectors/sql.py (1 hunks)
- pandasai/connectors/yahoo_finance.py (1 hunks)
- pandasai/constants.py (1 hunks)
- pandasai/engine.py (1 hunks)
- pandasai/helpers/anonymizer.py (1 hunks)
- pandasai/helpers/code_manager.py (1 hunks)
- pandasai/helpers/data_sampler.py (1 hunks)
- pandasai/helpers/df_info.py (1 hunks)
- pandasai/helpers/from_google_sheets.py (2 hunks)
- pandasai/pandas/init.py (1 hunks)
- pandasai/prompts/clarification_questions_prompt.py (1 hunks)
- pandasai/prompts/generate_python_code.py (2 hunks)
- pandasai/prompts/rephase_query_prompt.py (1 hunks)
- pandasai/responses/streamlit_response.py (1 hunks)
- pandasai/smart_dataframe/init.py (5 hunks)
- pandasai/smart_datalake/init.py (3 hunks)
- tests/unit_tests/test_smartdataframe.py (5 hunks)
Additional comments: 17
pandasai/engine.py (1)
- 8-24: The implementation of
set_pd_engineand_reload_pdfunctions inengine.pycorrectly handles engine switching with thread safety and input validation. Ensure that the reloading ofpandasai.pandasmodule is tested thoroughly to confirm that the engine switch takes effect immediately in all scenarios.pandasai/__init__.py (1)
- 26-31: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [8-30]
The addition of
set_pd_engineto the public interface ofpandasaiis correctly implemented, enhancing the module's flexibility by allowing users to switch betweenpandasandmodinengines. Ensure that the documentation is updated to guide users on how to use this new functionality effectively.pandasai/responses/streamlit_response.py (1)
- 3-3: Switching the import to
pandasai.pandasinstreamlit_response.pycorrectly aligns with the project's goal to supportmodin. This change ensures that DataFrame operations within theStreamlitResponseclass utilize the configured DataFrame engine, enhancing flexibility and performance.pandasai/prompts/rephase_query_prompt.py (1)
- 13-13: Changing the import to
pandasai.pandasinrephase_query_prompt.pyensures consistency across the project, allowing theRephraseQueryPromptclass to utilize the configured DataFrame engine. This modification supports the project's goal of integratingmodinseamlessly.pandasai/prompts/clarification_questions_prompt.py (1)
- 21-21: The modification to import
pandasai.pandasinclarification_questions_prompt.pyaligns with the project's objective to integratemodin. This change ensures that DataFrame operations within theClarificationQuestionPromptclass utilize the configured DataFrame engine, maintaining consistency and flexibility.pandasai/constants.py (1)
- 97-97: Adding "modin" to the list of dependencies in
constants.pycorrectly reflects the project's new dependency structure. This change is necessary for ensuring thatmodinis recognized as a valid dependency within the project, aligning with the PR's objectives.pandasai/connectors/sql.py (1)
- 15-15: The import statement has been changed to
import pandasai.pandas as pdto facilitate the use ofmodinfor parallel processing. Ensure that all DataFrame operations in this file are compatible with bothpandasandmodinto maintain the intended flexibility and performance improvements.pandasai/helpers/code_manager.py (1)
- 11-11: The import statement has been changed to
import pandasai.pandas as pdto facilitate the use ofmodinfor parallel processing. Verify that all DataFrame manipulations and operations withinCodeManagerand related functionalities are compatible with bothpandasandmodinto ensure seamless functionality.pandasai/smart_dataframe/__init__.py (3)
- 30-30: The import statement has been changed to
import pandasai.pandas as pdto facilitate the use ofmodinfor parallel processing. This change is foundational for enablingmodinsupport within theSmartDataframeclass.- 120-133: The
_load_enginemethod has been updated to recognizemodinas a valid engine. Ensure that all functionalities withinSmartDataframethat depend on the engine type are updated to handlemodinappropriately, including data manipulation and querying operations.- 391-406: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [394-424]
The
_truncate_head_columnsmethod has been updated to handle dataframes from themodinengine. This update is crucial for maintaining functionality when displaying or processing dataframe previews. Verify that the truncation logic works as expected withmodindataframes, especially sincemodinaims to mimicpandasbehavior closely.pandasai/smart_datalake/__init__.py (2)
- 73-73: The
_lockattribute is correctly initialized as an instance ofthreading.RLock, ensuring thread safety for directory creation operations.- 167-168: Using
self._lockwithin awithstatement to wrap theos.makedirscall ensures thread safety during directory creation. This is a good practice for preventing race conditions in a multi-threaded environment.tests/unit_tests/test_smartdataframe.py (4)
- 16-16: The import of
pandasaiand subsequent usage throughout the test file aligns with the PR's objective to replace directpandasreferences withpandasai.pandas. This change is crucial for ensuring that the tests reflect the new dependency management and dynamic import mechanism introduced by the PR.- 685-685: The use of
pandasai.pandas.read_parquetinstead of the standardpd.read_parquetis consistent with the PR's goal of integratingmodinsupport through thepandasai.pandasmodule. This ensures that the tests are aligned with the new import mechanism and validates the functionality withmodinas an optional dependency.- 723-723: Correctly using
pandasai.pandas.read_parquetfor testing the import of a Parquet file aligns with the PR's objectives. This ensures that the test suite is updated to reflect the new dynamic import mechanism, facilitating the integration ofmodinsupport.- 736-736: The use of
pandasai.pandas.read_excelfor testing the import of an Excel file is consistent with the PR's goal of integratingmodinsupport through thepandasai.pandasmodule. This change ensures that the test suite reflects the new import mechanism and validates the functionality withmodinas an optional dependency.
Codecov ReportAttention:
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #907 +/- ##
==========================================
- Coverage 84.89% 84.57% -0.32%
==========================================
Files 91 93 +2
Lines 3918 3973 +55
==========================================
+ Hits 3326 3360 +34
- Misses 592 613 +21 ☔ View full report in Codecov by Sentry. |
|
@gventuri You can start having a look if you like :) This PR is just missing a few tests. |
gventuri
reviewed
Feb 5, 2024
| @@ -10,7 +10,8 @@ | |||
| import random | |||
|
|
|||
| import numpy as np | |||
| import pandas as pd | |||
|
|
|||
| import pandasai.pandas as pd | |||
There was a problem hiding this comment.
So if I understand correctly, to make it work with modin we need to import pandas like this. While if we import with import pandas as pd it will still work, but just ignore the logic to optionally import it with modin, right?
There was a problem hiding this comment.
@mspronesti while if one, for example, import pandas as pd, it will still work, right? What was the issue with import modin.pandas as pd?
There was a problem hiding this comment.
@gventuri From the user perspective, pandasai is still supposed to be used with pandas or modin, i.e. one would just import one of the two in their snippet and use pandasai as they were before.
However, internally this import is needed because pandasai.pandas contains the logic to select the engine, which is triggered whenenver pandasai.set_pd_engine("modin|pandas") is called.
To sum up, from the user'sperspective:
- to use pandas: no changes
- to use modin:
pandasai.set_pd_engine("modin")and that's It - to use both in the same snippet but with different dataframes:
...
pandasai.set_pd_engine(modin|pandas)
# the selected engine will be used
...
pandasai.set_pd_engine(pandas|modin)
# now the new engine will be used
...By default, the engine is pandas thus if set_pd_engine is not called, pandas will be used. This makes pandasai fully backward compatible.
There was a problem hiding this comment.
@mspronesti thanks a lot for clarifying, makes a lot of sense! I'll have a final review later today and merge it! Thanks a lot for the great improvement, super helpful for larger datasets!
There was a problem hiding this comment.
PR Type: Enhancement
PR Summary: This pull request introduces support for Modin dataframes in the PandasAI library, allowing users to switch between Pandas and Modin engines dynamically. It includes modifications across various modules to accommodate the new dataframe engine, updates to documentation and examples to guide users on how to use the new feature, and adjustments to the testing suite to ensure compatibility. Additionally, the PR restructures dependencies to make 'dev' and 'docs' optional, aiming to streamline the installation process.
Decision: Comment
📝 Type: 'Enhancement' - not supported yet.
- Sourcery currently only approves 'Typo fix' PRs.
✅ Issue addressed: this change correctly addresses the issue or implements the desired feature.
No details provided.
📝 Complexity: the changes are too large or complex for Sourcery to approve.
- Unsupported files: the diff contains files that Sourcery does not currently support during reviews.
General suggestions:
- Ensure that the documentation is clear and comprehensive, especially regarding the dynamic switching between Pandas and Modin engines, to help users understand how to effectively use this new feature.
- Consider adding more detailed examples in the documentation that showcase the performance benefits or use cases where switching to Modin might be particularly advantageous.
- Verify that all modified connectors and helpers function as expected with both Pandas and Modin engines, particularly in edge cases or with large datasets.
- Review the typo fix in the installation command for the Modin dependency in the examples documentation to ensure accuracy and clarity for users following the guide.
Thanks for using Sourcery. We offer it for free for open source projects and would be very grateful if you could help us grow. If you like it, would you consider sharing Sourcery on your favourite social media? ✨
There was a problem hiding this comment.
Review Status
Actionable comments generated: 3
Configuration used: CodeRabbit UI
Files ignored due to path filters (2)
poetry.lockis excluded by:!**/*.lockpyproject.tomlis excluded by:!**/*.toml
Files selected for processing (10)
- .github/workflows/ci.yml (1 hunks)
- CONTRIBUTING.md (1 hunks)
- docs/examples.md (1 hunks)
- docs/getting-started.md (1 hunks)
- pandasai/connectors/airtable.py (1 hunks)
- pandasai/helpers/df_config_manager.py (1 hunks)
- pandasai/helpers/df_validator.py (1 hunks)
- pandasai/smart_dataframe/abstract_df.py (1 hunks)
- tests/unit_tests/test_df_info.py (3 hunks)
- tests/unit_tests/test_smartdataframe.py (7 hunks)
Files skipped from review due to trivial changes (1)
- CONTRIBUTING.md
Additional comments: 13
.github/workflows/ci.yml (1)
- 25-25: The addition of
--with devto thepoetry installcommand is approved. Ensure to monitor CI build times and resource usage to avoid potential inefficiencies.tests/unit_tests/test_df_info.py (2)
- 15-17: Creating a Modin DataFrame for testing is approved. Ensure the data structure and values are consistent with the test's objectives.
- 35-38: The new test method
test_df_type_modinis approved. Ensure it accurately tests the Modin DataFrame type recognition.pandasai/helpers/df_validator.py (1)
- 104-104: The addition of "modin" support in
_df_to_list_of_dictis approved. Ensure comprehensive testing to validate Modin DataFrame handling and that existing functionality remains unaffected.pandasai/helpers/df_config_manager.py (1)
- 81-81: The extension of Parquet file support for "modin" dataframes is approved. Ensure testing for correctness of the generated Parquet files from Modin DataFrames.
pandasai/smart_dataframe/abstract_df.py (1)
- 300-300: The addition of "modin" support in the
to_dictmethod is approved. Ensure comprehensive testing to validate correct processing of Modin DataFrames and that existing functionality remains unaffected.pandasai/connectors/airtable.py (1)
- 13-13: The change to import
pandasai.pandasis approved. Ensure testing to validate that the Airtable connector functions correctly with both Modin and standard Pandas DataFrames.docs/getting-started.md (1)
- 30-30: The addition of
modinto the optional dependencies documentation is approved. Ensure consistency in the documentation.tests/unit_tests/test_smartdataframe.py (5)
- 335-348: The addition of
modinas an engine type for theto_dictmethod tests is correctly implemented. However, ensure that themodinengine is properly initialized and available in the test environment to avoid runtime errors.- 692-692: Mocking
pandasai.pandas.read_parquetfor loading dataframes from saved definitions is correctly done. Ensure that the mocked return value matches the expected dataframe structure for accurate test results.- 707-707: The test method
test_load_dataframe_from_polarscorrectly checks the loading of a Polars dataframe into the SmartDataframe. This ensures compatibility with Polars dataframes alongside Pandas and Modin.- 730-730: The test method
test_import_parquet_filecorrectly mockspandasai.pandas.read_parquetfor testing the import of Parquet files. Ensure that the mocked return value accurately represents the expected dataframe structure.- 743-743: The test method
test_import_excel_filecorrectly mockspandasai.pandas.read_excelfor testing the import of Excel files. This ensures that Excel file imports are properly handled by the SmartDataframe class.
| @@ -1,5 +1,6 @@ | |||
| import unittest | |||
|
|
|||
| import modin.pandas as mpd | |||
There was a problem hiding this comment.
Ensure Modin is installed before running tests that require it, to avoid test failures in environments without Modin.
import pytest
@pytest.mark.skipif(not modin_installed, reason="Modin is not installed")
There was a problem hiding this comment.
Review Status
Actionable comments generated: 5
Configuration used: CodeRabbit UI
Files selected for processing (2)
- docs/examples.md (1 hunks)
- pandasai/smart_dataframe/init.py (10 hunks)
Files skipped from review as they are similar to previous changes (1)
- docs/examples.md
Additional comments: 3
pandasai/smart_dataframe/__init__.py (3)
- 118-133: The
_load_enginemethod's error message in line 129-130 could be misleading ifpd.__name__is dynamically set to either 'pandas' or 'modin'. Ensure the message accurately reflects the current engine being used, especially sincepdis an alias that could point to either library based on configuration.- 127-127: The check for 'df_engine != "polars"' seems redundant since 'df_type' will never return 'polars' if 'self._df' is not a Polars DataFrame. Simplify the condition to just check the instance of 'self._df'.
- 395-395: The 'engine' variable is used before its assignment. This could lead to a 'NameError' if the 'engine' variable is not defined in the scope where '_truncate_head_columns' is called. Ensure that 'engine' is defined before this check.
|
@mspronesti thanks a lot for the great PR, merging! |
gventuri
added a commit
that referenced
this pull request
Feb 20, 2024
* fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (#889) (#890) * refactor: TypeVar for IResponseParser (#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (#901) * fix: badge for "Open in Colab" (#903) * feat: add support Google Gemini API in LLMs (#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]>
gventuri
added a commit
that referenced
this pull request
Feb 20, 2024
… 1 message (#947) * fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (#889) (#890) * refactor: TypeVar for IResponseParser (#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (#901) * fix: badge for "Open in Colab" (#903) * feat: add support Google Gemini API in LLMs (#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]>
gventuri
added a commit
that referenced
this pull request
Feb 21, 2024
* fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (#889) (#890) * refactor: TypeVar for IResponseParser (#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (#901) * fix: badge for "Open in Colab" (#903) * feat: add support Google Gemini API in LLMs (#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default * fix: function call check and query tracker tracking --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]>
gventuri
added a commit
that referenced
this pull request
Feb 29, 2024
* refactor: clean code in smart_dataframe and smart_datalake (#814) * refactor: extract import from file method * refactor: extract df head methods * refactor: move connector config in the relative connector file * refactor: csv and pandas files are now treated as a connector * chore: remove verbose getters and setters * refactor: remove load and save feature * refactor: create dataframe proxy * chore: simplify agent * chore: simplify datalake * refactor: simplify smart datalake * refactor: centralize context in lakes * refactor: move lake callbacks to dedicate class * fix: load connector before generating cache hex * fix: only allow direct sql to SQLConnectors * fix: check sql connector was not working * fix(connector): update connector validation at the start * fix(direct_sql): fix some leftovers * fix: merged change revert built-in shadowing --------- Co-authored-by: ArslanSaleem <[email protected]> * refactor(query_tracker): clean code and create error handling pipeling (#875) * refactor(smart_datalake): clean chat method of smart datalake * refactor(smartlake_pipeline): minor cleanups and comments * refactor(query_tracker): refactor query tracker input and output * refactor(query_tracker): adding error pipeline * refactor(query_tracker): some rename and delete extras * refactor(query_tracker): remove comments * Merge Main Changes to v1.6 (#878) * fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> * refactor: remove inheritance of pandas methods in the SmartDataframe * refactor: remove shortcuts * refactor: smart df cannot chat anymore * refactor: remove unused validate method from sdf * refactor: add name and description to connectors * refactor: remove custom prompts * refactor: remove synthetic pipelines * refactor: remove custom instructions * refactor: remove starcoder and falcon * refactor: move df head to connectors * refactor: remove sdf * refactor: remove leftovers logic units * refactor: rename SmartDataLake to AgentCore * refactor: rename DataLake pipeline to Chat * refactor: remove agent core, moving the logic to agent * refactor: refactor the prompt templates using Jinja2 * feat(Agent): train agent docs or question/answers (#895) * feat(VectorStore): adding chromadb vector store for RAG * test(agent_train): adding more test cases and error handling * refact(agent_train): add logging to chroma db * feat(RAG): use trained data from vector db in prompt (#896) * feat(VectorStore): adding chromadb vector store for RAG * test(agent_train): adding more test cases and error handling * refact(agent_train): add logging to chroma db * feat(RAG): use trained vector in prompt * Merge v1.6 into to v2.0 (#900) * refactor(Prompt): adding more context for dataframe and multiple types of serialization (#880) * refactor(prompt): update dataframe serialization in prompt * tests(prompt): fix and adding new tests * fix(prompt): adding type to dataframe serialize function * refactor(prompt): add field descriptions in yml prompt * fix(prompt): direct_sql still using old dataframe table * refactor(direct_sql): add instruction and note for using relevant table only * refactor(query_tracker): clean output of query tracker format (#888) * feat(GoogleBigQuery): adding google big query connector (#886) * feat(VectorStore): adding chromadb vector store for RAG * test(agent_train): adding more test cases and error handling * refact(agent_train): add logging to chroma db * feat(RAG): use trained vector in prompt * Merge v1.6 to v2.0 * feat: execute_sql_query_usage creating error prompt if execute_sql_que… (#898) * feat/execute_sql_query_usage creating error prompt if execute_sql_query is not used when direct_sql is set to true * fix/comment added before raising exception --------- Co-authored-by: Milind Lalwani <[email protected]> * fix(prompt_path): path issue if pandasai used outside of pandas env (#909) * feat(agent): adding optional pipeline call (#916) s * feat(system prompt): use system in llm prompts (#925) * fix(code_manager): minor fixes in code manager * feat(system_prompt): adding system prompts to be added in llm call * fix(code_manager): minor fixes in code manager (#923) Co-authored-by: Gabriele Venturi <[email protected]> * feat/update method in vector db added * feat/get by id method added * feat(multiturn-conv): update prompts and use api's for multiturn (#928) * fix(code_manager): minor fixes in code manager * feat(system_prompt): adding system prompts to be added in llm call * feat(multi-turn-conv): adding support for multi turn conversation * feat(multi-turn): add missing files * feat(BambooVectorStore): adding bamboo vector to store and retrieve t… (#935) * feat(BambooVectorStore): adding bamboo vector to store and retrieve training docs on cloud * rename test class name * feat(BambooLLM): add bamboo llm wrapper (#940) * feat(BambooVectorStore): adding bamboo vector to store and retrieve training docs on cloud * rename test class name * feat(bamboo_llm): adding bamboo llm interface * fix: clean up few commented and leftover changes * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * Merge to release 2.0 (#945) * fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (#889) (#890) * refactor: TypeVar for IResponseParser (#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (#901) * fix: badge for "Open in Colab" (#903) * feat: add support Google Gemini API in LLMs (#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default * fix(skills): add skills to query tracker and by default rag to return 1 message (#947) * fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (#889) (#890) * refactor: TypeVar for IResponseParser (#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (#901) * fix: badge for "Open in Colab" (#903) * feat: add support Google Gemini API in LLMs (#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> * fix: function call check and query tracker tracking * fixes/in release2.0 (#951) * fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (#889) (#890) * refactor: TypeVar for IResponseParser (#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (#901) * fix: badge for "Open in Colab" (#903) * feat: add support Google Gemini API in LLMs (#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default * fix: function call check and query tracker tracking --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> * update comparison operator (#953) * docs: improve readme and license * lint: fix lint in examples * docs: add deploy methods in the documentation * refactor: rename to ChromaDB * feat: make BambooVector the default vectorstore * lint: fix lint in examples * docs: add docs for training, agent description * docs: add video for training LLM --------- Co-authored-by: ArslanSaleem <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]>
gventuri
added a commit
that referenced
this pull request
Mar 8, 2024
* Release v2.0 * refactor: clean code in smart_dataframe and smart_datalake (#814) * refactor: extract import from file method * refactor: extract df head methods * refactor: move connector config in the relative connector file * refactor: csv and pandas files are now treated as a connector * chore: remove verbose getters and setters * refactor: remove load and save feature * refactor: create dataframe proxy * chore: simplify agent * chore: simplify datalake * refactor: simplify smart datalake * refactor: centralize context in lakes * refactor: move lake callbacks to dedicate class * fix: load connector before generating cache hex * fix: only allow direct sql to SQLConnectors * fix: check sql connector was not working * fix(connector): update connector validation at the start * fix(direct_sql): fix some leftovers * fix: merged change revert built-in shadowing --------- Co-authored-by: ArslanSaleem <[email protected]> * refactor(query_tracker): clean code and create error handling pipeling (#875) * refactor(smart_datalake): clean chat method of smart datalake * refactor(smartlake_pipeline): minor cleanups and comments * refactor(query_tracker): refactor query tracker input and output * refactor(query_tracker): adding error pipeline * refactor(query_tracker): some rename and delete extras * refactor(query_tracker): remove comments * Merge Main Changes to v1.6 (#878) * fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> * refactor: remove inheritance of pandas methods in the SmartDataframe * refactor: remove shortcuts * refactor: smart df cannot chat anymore * refactor: remove unused validate method from sdf * refactor: add name and description to connectors * refactor: remove custom prompts * refactor: remove synthetic pipelines * refactor: remove custom instructions * refactor: remove starcoder and falcon * refactor: move df head to connectors * refactor: remove sdf * refactor: remove leftovers logic units * refactor: rename SmartDataLake to AgentCore * refactor: rename DataLake pipeline to Chat * refactor: remove agent core, moving the logic to agent * refactor: refactor the prompt templates using Jinja2 * feat(Agent): train agent docs or question/answers (#895) * feat(VectorStore): adding chromadb vector store for RAG * test(agent_train): adding more test cases and error handling * refact(agent_train): add logging to chroma db * feat(RAG): use trained data from vector db in prompt (#896) * feat(VectorStore): adding chromadb vector store for RAG * test(agent_train): adding more test cases and error handling * refact(agent_train): add logging to chroma db * feat(RAG): use trained vector in prompt * Merge v1.6 into to v2.0 (#900) * refactor(Prompt): adding more context for dataframe and multiple types of serialization (#880) * refactor(prompt): update dataframe serialization in prompt * tests(prompt): fix and adding new tests * fix(prompt): adding type to dataframe serialize function * refactor(prompt): add field descriptions in yml prompt * fix(prompt): direct_sql still using old dataframe table * refactor(direct_sql): add instruction and note for using relevant table only * refactor(query_tracker): clean output of query tracker format (#888) * feat(GoogleBigQuery): adding google big query connector (#886) * feat(VectorStore): adding chromadb vector store for RAG * test(agent_train): adding more test cases and error handling * refact(agent_train): add logging to chroma db * feat(RAG): use trained vector in prompt * Merge v1.6 to v2.0 * feat: execute_sql_query_usage creating error prompt if execute_sql_que… (#898) * feat/execute_sql_query_usage creating error prompt if execute_sql_query is not used when direct_sql is set to true * fix/comment added before raising exception --------- Co-authored-by: Milind Lalwani <[email protected]> * fix(prompt_path): path issue if pandasai used outside of pandas env (#909) * feat(agent): adding optional pipeline call (#916) s * feat(system prompt): use system in llm prompts (#925) * fix(code_manager): minor fixes in code manager * feat(system_prompt): adding system prompts to be added in llm call * fix(code_manager): minor fixes in code manager (#923) Co-authored-by: Gabriele Venturi <[email protected]> * feat/update method in vector db added * feat/get by id method added * feat(multiturn-conv): update prompts and use api's for multiturn (#928) * fix(code_manager): minor fixes in code manager * feat(system_prompt): adding system prompts to be added in llm call * feat(multi-turn-conv): adding support for multi turn conversation * feat(multi-turn): add missing files * feat(BambooVectorStore): adding bamboo vector to store and retrieve t… (#935) * feat(BambooVectorStore): adding bamboo vector to store and retrieve training docs on cloud * rename test class name * feat(BambooLLM): add bamboo llm wrapper (#940) * feat(BambooVectorStore): adding bamboo vector to store and retrieve training docs on cloud * rename test class name * feat(bamboo_llm): adding bamboo llm interface * fix: clean up few commented and leftover changes * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * Merge to release 2.0 (#945) * fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (#889) (#890) * refactor: TypeVar for IResponseParser (#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (#901) * fix: badge for "Open in Colab" (#903) * feat: add support Google Gemini API in LLMs (#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default * fix(skills): add skills to query tracker and by default rag to return 1 message (#947) * fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (#889) (#890) * refactor: TypeVar for IResponseParser (#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (#901) * fix: badge for "Open in Colab" (#903) * feat: add support Google Gemini API in LLMs (#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> * fix: function call check and query tracker tracking * fixes/in release2.0 (#951) * fix(output_type): handle errors for wrong output type (#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (#876) * fix: rephrase query (#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (#889) (#890) * refactor: TypeVar for IResponseParser (#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (#901) * fix: badge for "Open in Colab" (#903) * feat: add support Google Gemini API in LLMs (#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default * fix: function call check and query tracker tracking --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> * update comparison operator (#953) * docs: improve readme and license * lint: fix lint in examples * docs: add deploy methods in the documentation * refactor: rename to ChromaDB * feat: make BambooVector the default vectorstore * lint: fix lint in examples * docs: add docs for training, agent description * docs: add video for training LLM --------- Co-authored-by: ArslanSaleem <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> * feat: fix poetry.lock * fix: update domain name (#970) * docs: improve docs about training * fix(agent): langchain LLM instantiation (#977) * fix(agent): langchain LLM instantiation * replace object with mock * feat: add support for Gemini API * Release v2.0.2 * fix(llm): Google Gemini signature and memory usage (#980) * Release v2.0.3 * Release v2.0.3 * build: fix issue * build: fix ci for windows (#1005) * fix: add number result to memory as string (#997) Co-authored-by: Gabriele Venturi <[email protected]> * fix: remove logger refs from SmartDataframe (#985) * Remove logger from smart_dataframe constructor * Remove logger from SmartDatalake args definition --------- Co-authored-by: Gabriele Venturi <[email protected]> * chore(code_manager): filter code for redeclaration of pd.DataFrame from head (#1003) * feat(CockroachDBConnector): add support for cockroachdb * fix: cockroach db connector * fix: import polars (#1009) * Update polars dependency Move polars dependency into init to make it optional * fix: polars import * fix: fix linter --------- Co-authored-by: PVA <[email protected]> * Release v2.0.4 * Release v2.0.5 --------- Co-authored-by: ArslanSaleem <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> Co-authored-by: Cheng Wai <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi @gventuri,
this draft PR finally adds support for
modin! Please take your time to review it very carefully, as it introduces some core changes you should have a look at. Differently from my initial thought, this feature was not trivial at all.In addition, I have re-structured the dependencies to make
devanddocsoptional (beforepoetry installinstalled all of them).I still have to update the docs and add some tests.
Usage
Summary by CodeRabbit
New Features
smart_dataframemodule to support multiple data processing engines such as Pandas, Modin, and Polars.chatmethod to support various output types like pandas, modin, and polars dataframes.Refactor
Documentation