Adds python_exec into ImageSpec

[thomasjpfan]

Why are the changes needed?

Currently, ImageSpec will always create a new virtualenv to install packages. With this PR, users can specify, python_exec to use a Python from a base_image's python to install packages into.

What changes were proposed in this pull request?

This PR:

• Moves the micromamba python install into it's own function and only uses it if python_exec is not set.
• Errors in envd when python_exec is set.
• Errors when any of the conda_* parameters are set and python_exec is set.

How was this patch tested?

I ran the follow on sandbox:

from flytekit import ImageSpec, task

image = ImageSpec(
    base_image="ghcr.io/flyteorg/flytekit:py3.11-1.14.4",
    name="wow",
    apt_packages=["git"],
    packages=[
        "numpy",
        "git+https://github.com/thomasjpfan/flytekit.git@176ecfbe006e00d563380d4c2638c3e550b35fbc",
    ],
    registry="localhost:30000",
    python_exec="/usr/local/bin/python",
)


@task(container_image=image)
def main() -> float:
    import numpy as np

    return np.ones(10).sum().item()

And confirm that the image built uses system python.

Summary by Bito

This PR introduces a new ImageSpec feature allowing custom Python executable specification for package installation, replacing the default virtualenv approach. The implementation includes Micromamba installation logic refactoring, validation for conda-related parameters, and proper error handling in the envd builder. The changes are well-structured with comprehensive validation checks for python_exec paths.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 2

[flyte-bot]

Code Review Agent Run #d07b90

Actionable Suggestions - 4

• flytekit/image_spec/image_spec.py - 1

  • Consider adding validation for python_exec · Line 91-91
• plugins/flytekit-envd/flytekitplugins/envd/image_builder.py - 1

  • Consider more descriptive error message · Line 101-102
• flytekit/image_spec/default_builder.py - 1

  • Consider adding python_exec path validation · Line 254-257
• tests/flytekit/unit/core/image_spec/test_default_builder.py - 1

  • Consider expanding python_exec validation test coverage · Line 344-344

Review Details

• Files reviewed - 5 · Commit Range: 176ecfb..176ecfb

  • flytekit/image_spec/default_builder.py
  • flytekit/image_spec/image_spec.py
  • plugins/flytekit-envd/flytekitplugins/envd/image_builder.py
  • plugins/flytekit-envd/tests/test_image_spec.py
  • tests/flytekit/unit/core/image_spec/test_default_builder.py
• Files skipped - 0
• Tools

  • Whispers (Secret Scanner) - ✔︎ Successful
  • Detect-secrets (Secret Scanner) - ✔︎ Successful
  • MyPy (Static Code Analysis) - ✔︎ Successful
  • Astral Ruff (Static Code Analysis) - ✔︎ Successful

---

AI Code Review powered by

[flyte-bot]

Changelist by Bito

This pull request implements the following key changes.

Key Change

Files Impacted

Feature Improvement - Enhanced Python Environment Configuration in ImageSpec

- default_builder.py - Added support for custom Python executable and refactored Micromamba installation logic - image_spec.py - Added python_exec parameter to ImageSpec class - image_builder.py - Added validation for python_exec in envd builder - test_image_spec.py - Added tests for python_exec validation in envd builder - test_default_builder.py - Added tests for python_exec functionality in default builder

[codecov]

Codecov Report

Attention: Patch coverage is 94.59459% with 2 lines in your changes missing coverage. Please review.

Project coverage is 76.85%. Comparing base (30088c2) to head (176ecfb).
Report is 8 commits behind head on master.

Files with missing lines	Patch %	Lines
flytekit/image_spec/default_builder.py	92.59%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3069      +/-   ##
==========================================
- Coverage   82.79%   76.85%   -5.95%     
==========================================
  Files           3      205     +202     
  Lines         186    21602   +21416     
  Branches        0     2750    +2750     
==========================================
+ Hits          154    16602   +16448     
- Misses         32     4239    +4207     
- Partials        0      761     +761     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[eapolinario]

The greatexpectations failures are unrelated.