-
Notifications
You must be signed in to change notification settings - Fork 89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs : docstrings examples - dtypes #1121
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks @anopsy !
for Categorical, it looks like casting from string is unsupported in old pyarrow versions* - but you can create the categorical directly:
>>> ca = pa.chunked_array(pa.array(['a', 'b'], type=pa.dictionary(pa.uint32(), pa.string())))
>>> nw.from_native(ca, series_only=True)
┌─────────────────────────────────────────┐
| Narwhals Series |
| Use `.to_native()` to see native output |
└─────────────────────────────────────────┘
>>> nw.from_native(ca, series_only=True).dtype
Categorical
For Struct / List / Array, there is a way to do that in pandas too use pd.ArrowDtype
:
In [11]: pd.Series(data, dtype=pd.ArrowDtype(pa.large_list(pa.large_string())))
Out[11]:
0 ['narwhal' 'orca']
1 ['beluga' 'vaquita']
dtype: large_list<item: large_string>[pyarrow]
Similarly, for the struct example, I think you could use
pa.struct({'a': pa.int64(), 'b': pa.large_list(pa.large_string())})
*maybe we should only run doctests on the latest versions...that would save us a lot of # doctest: +SKIP
s which would be a good thing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for updating!
tests/stable_api_test.py
Outdated
@@ -12,6 +12,33 @@ | |||
from tests.utils import Constructor | |||
from tests.utils import assert_equal_data | |||
|
|||
DTYPES = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just out of interest, why add this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hey Marco, I wasn't updating, I was just trying to resolve some conflicts and update my fork/local repo. I was working on the dtypes file and also had to adjust the tests and then you did some changes to the test file (among those was adding this DTYPES = {....}-thing, I think you did it during one of the live streams.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for resolving merge conflicts then 🙌
i think it might not be necessary any more to have DTYPES
here then, could you try removing it from this file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think I first merged your changes and then committed them back, in the process of updating my local repo. I can see now that you refactored this test 2 weeks ago or so. Yes, I'll try to get it right!
narwhals/dtypes.py
Outdated
>>> ser_pl = pl.Series(data) | ||
>>> ser_pa = pa.chunked_array([data]) | ||
|
||
>>> nw.from_native(ser_pl, series_only=True).dtype | ||
List(String) | ||
>>> nw.from_native(ser_pa, series_only=True).dtype | ||
List(String) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if you want a pandas example we could add
pd.Series(data, dtype=pd.ArrowDtype(pa.large_list(pa.large_string())))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, it's done, although I did --force push after rebasing, I hope I didn't break anything. If everything is fine, could we just merge this PR and I will continue from a new branch because I butchered this one so much 😅
Just a quick check: considering I'll have to use cast to get some of the datatypes, maybe I should wrap the from_native into a func after all?
What type of PR is this? (check all applicable)
Related issues
Checklist
If you have comments or can explain your changes, please do so below.