Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(python): dont show wrong inefficient apply warning when taking numpy func of constant #10103

Closed
wants to merge 3 commits into from

Conversation

MarcoGorelli
Copy link
Collaborator

@MarcoGorelli MarcoGorelli commented Jul 26, 2023

I'd missed this when reviewing, but a "nonsense" warning is currently shown in some cases:

df = pl.DataFrame({"a": [1, 2, 3]})
df.select(pl.col("a").apply(lambda x: np.sin(3) + x))

shows

-  pl.col("a").apply(lambda x: ...)
+  3.sin() + pl.col("a")

I think we need to be extra-defensive here, and match as explicitly as we can - I've added param_name_index to the function so we can check that parameter name is exactly in the position we expect it to be


with this PR, the code above wouldn't display any warning, as np.sin(3) isn't a simple-enough constant to be considered (yet!)

@github-actions github-actions bot added fix Bug fix python Related to Python Polars labels Jul 26, 2023
@MarcoGorelli MarcoGorelli marked this pull request as ready for review July 26, 2023 20:56
block_offsets = list(expression_blocks.keys())
block_offsets.remove(start)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -113,6 +113,13 @@
("c", lambda x: json.loads(x), 'pl.col("c").str.json_extract()'),
]

NOOP_TEST_CASES = [
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moving these here from test_inefficient_apply, we should probably be testing the "no-op" cases across all supported python versions too

@alexander-beedie
Copy link
Collaborator

alexander-beedie commented Jul 27, 2023

Hmm, turns out it only takes two/three lines to support this properly; as the result will resolve to a const it can be integrated more or less 'as is' - see what you think? 😅 Commit: f44b1ed.

Tried it out with lambda x: numpy.cos(3) + x - abs(-1).

(Unrelated change: removed len for now as it needs to be mapped to str.lengths).

@MarcoGorelli
Copy link
Collaborator Author

well you're a bit good, aren't you? 🙌

@MarcoGorelli
Copy link
Collaborator Author

closing in favour of #10104

@alexander-beedie
Copy link
Collaborator

alexander-beedie commented Jul 27, 2023

well you're a bit good, aren't you?

I think you mean "a bit OCD" ... 🤣

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix Bug fix python Related to Python Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants