Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set operations do not work with pl.lit, only with dataframe columns #9764

Closed
2 tasks done
moritzwilksch opened this issue Jul 7, 2023 · 5 comments
Closed
2 tasks done
Labels
bug Something isn't working python Related to Python Polars

Comments

@moritzwilksch
Copy link
Contributor

moritzwilksch commented Jul 7, 2023

Polars version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of Polars.

Issue description

The newly added set operations seem to have problems with using pl.lit(). They only work as expected on other dataframe columns.

Reproducible example

import polars as pl

df = pl.DataFrame({"a": [[1, 2, 3], [4, 5]]})

df.with_columns(b=pl.lit([[3, 4, 6]])).select(pl.col("a").list.intersection("b"))
# shape: (2, 1)
# ┌───────────┐
# │ a         │
# │ ---       │
# │ list[i64] │
# ╞═══════════╡
# │ [3]       │
# │ [4]       │
# └───────────┘

df.select(pl.col("a").list.intersection(pl.lit([[3, 4, 6]])))
# PanicException: assertion failed: `(left == right)`
#   left: `3`,
#  right: `2`

Expected behavior

The behavior should be identical between using a literal expression or an existing dataframe column.

Installed versions

--------Version info---------
Polars:              0.18.6
Index type:          UInt32
Platform:            Linux-5.4.0-1103-aws-x86_64-with-glibc2.31
Python:              3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0]

----Optional dependencies----
adbc_driver_sqlite:  <not installed>
connectorx:          <not installed>
deltalake:           <not installed>
fsspec:              <not installed>
matplotlib:          3.7.1
numpy:               1.21.6
pandas:              1.5.3
pyarrow:             12.0.0
pydantic:            1.10.9
sqlalchemy:          1.4.48
xlsx2csv:            <not installed>
xlsxwriter:          <not installed>
@moritzwilksch moritzwilksch added bug Something isn't working python Related to Python Polars labels Jul 7, 2023
@deanm0000
Copy link
Collaborator

This is intended, see here

@moritzwilksch
Copy link
Contributor Author

Hmm fair, but then I would expect this to behave similarly to is_in which can also take literal lists as arguments, no?

df.select(pl.col("a").list.intersection([3, 4, 6]))
# PanicException: assertion failed: `(left == right)`
#  left: `3`,
#  right: `2`

@deanm0000
Copy link
Collaborator

I'm not following you.

@ritchie46
Copy link
Member

No, the literal should broadcast in this case. As it is an argument to the set function. That's within our broadcasting rules.

@orlp
Copy link
Collaborator

orlp commented Sep 4, 2023

This was solved in #10668.

@orlp orlp closed this as completed Sep 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working python Related to Python Polars
Projects
None yet
Development

No branches or pull requests

4 participants