Add basic support for gemini models

[ssonal]

Preliminary support for gemini.

What's currently missing:

• Async
• Tool use

---

🚀 This description was created by Ellipsis for commit 25c28bb

Summary:

This PR introduces basic support for Gemini models in the Instructor library, including a new client, updated documentation, and test cases.

Key points:

• Added basic support for Gemini models in the Instructor library.
• Introduced a new client for handling Gemini model interactions.
• Updated documentation and examples to include Gemini model usage.
• Added test cases for Gemini model functionality.
• Modified retry logic and response handling to accommodate Gemini-specific features.

---

Generated with ❤️ by ellipsis.dev

[sweep-ai]

Sweep: PR Review

Sweep is currently reviewing your pr...

[jxnl]

can you add a note in the patching notes

[jxnl]

@ellipsis-dev can you try and update the documentation on patch.md docs

[ellipsis-dev]

@jxnl, I have addressed your comments in pull request #687

[jxnl]

Can you jsut add docs to patch.md and the main readme

[ssonal]

Will try and get these in today @jxnl.

Also came across #624, and it's best if we don't have to introduce new syntax. Taking a stab at standardising Gemini api calls as well.

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to 25c28bb in 3 minutes and 17 seconds

More details

• Looked at 455 lines of code in 13 files
• Skipped 1 files when reviewing.
• Skipped posting 2 drafted comments based on config settings.

1. instructor/client_gemini.py:9

• Draft comment:
  The current implementation lacks support for asynchronous operations, which might be necessary for non-blocking I/O environments. Consider adding async support to align with modern Python practices and to ensure compatibility with async frameworks.
• Reason this comment was not posted:
  Confidence changes required: 66%
  The PR description mentions that async, partials/streaming, and tool use are missing. The code provided in the PR does not include any async handling, which is consistent with the description. However, it's important to note that the absence of async support might limit the usability of the gemini integration in environments that require non-blocking operations.

2. pyproject.toml:32

• Draft comment:
  Please ensure that the new documentation files (docs/concepts/patching.md and docs/index.md) are added to mkdocs.yml to make them accessible through the MkDocs site.
• Reason this comment was not posted:
  Confidence of 0% on close inspection, compared to threshold of 85%.

Workflow ID: wflow_yjUW9cC51ol7iTUd

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[Cruppelt]

Out of curiosity - Is there a reason this isn't using the official function calling API? https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling

[ssonal]

Out of curiosity - Is there a reason this isn't using the official function calling API? https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling

I have function calling on a branch but the responses are inconsistent and the response payload structure varies with every call. Not reliable enough to recommend at this time.

[jxnl]

1. are we ready to merge
2. lets move evals to https://github.com/instructor-ai/evals at some point, so lets not worry about that

[ssonal]

Yes this is good to go!

[AriMKatz]

Can this do multi modal ?

[jxnl]

whoo @ssonal found a bug in imports #697