docs(python): More accurate and helpful docs for user defined functions

[itamarst]

Fixes #14699
Fixes #14508

1. Focuses purely on Python, as in the title of the article, with Rust APIs moved elsewhere.
2. Doesn't give erroneous information about the limits of map_batches(), given implementation changes.
3. Doesn't mention map_elements(), for simplicity's sake. This can eventually be added in a new section, presuming I can figure out why map_elements() even exists.
4. Adds demonstration of how to write fast user functions.
5. Moved around some docs (using struct for multi-column support).
6. Tiny updates to NumPy docs, bit redundant but that is fine probably.

This adds Numba as documentation requirement, so when e.g. Python 3.13 is released, docs won't be buildable on 3.13 until Numba has a new release.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.41%. Comparing base (332e40a) to head (ca8ba41).

❗ Current head ca8ba41 differs from pull request most recent head 0778a44

Please upload reports for the commit 0778a44 to get more accurate results.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #15194      +/-   ##
==========================================
+ Coverage   80.81%   81.41%   +0.59%     
==========================================
  Files        1464     1425      -39     
  Lines      192019   187964    -4055     
  Branches     2743     2704      -39     
==========================================
- Hits       155185   153027    -2158     
+ Misses      36323    34440    -1883     
+ Partials      511      497      -14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[MarcoGorelli]

Thanks for working on this!

Doesn't mention map_elements(), for simplicity's sake. This can eventually be added in a new section, presuming I can figure out why map_elements() even exists.

The purpose of map_elements is for elementwise UDFs. If you're calling a 3rd party library (e.g. libpostal) which works on one element at a time, then that's a good use case for map_elements. It's a last-resort, really, but I think it does need mentioning for those cases where nothing else will do. Polars can also (sometimes) warn the user when they could have done better 😎

Regarding Numba: given that is' quite dangerous to use it when the Series has missing values, I don't think it should currently be recommended in the user guide. I'd suggest either keeping it out for now, or sorting out Numba first

[itamarst]

Merged forward, ready for review again @MarcoGorelli.

[MarcoGorelli]

thanks, looking better

I think map_elements needs mentioning though. perhaps you could even start with this one? Something like:

1. show that you can do pl.col('a').map_elements(lambda x: math.log(x))
2. then, say that it's better to use vectorised operations, and if you have a numpy function you want to apply, you can pass the column all in at once with pl.col('a').map_batches(np.log)
3. then, show how you can use numba to pass a custom vectorised function

[itamarst]

Thank you, I will address these.

[itamarst]

@MarcoGorelli back to you, either addressed or commented above.

[MarcoGorelli]

Seeing

FAILED tests/docs/test_user_guide.py::test_run_python_snippets[path10] - polars.exceptions.PolarsInefficientMapWarning: 
Expr.map_elements is significantly slower than the native expressions API.
Only use if you absolutely CANNOT implement your logic otherwise.
Replace this expression...
  - pl.col("values").map_elements(my_log)
with this one instead:
  + pl.col("values").log()

pop up in the CI logs brought a smile to my face 😄

[MarcoGorelli]

Thanks @itamarst , good one!

I made some minor comments, and made a commit to address them, as they're really minor

Leaving open a little in case others have objections, will merge if they don't

---

EDIT: I can't merge this one, but I think it's ready

[itamarst]

@MarcoGorelli I merged forward to deal with conflicts.

[itamarst]

@MarcoGorelli can we get this merged?

[ritchie46]

Can you do a rebase?

[itamarst]

Will do that now.

[itamarst]

My editor is having Issues so still need to manually verify that merge was correct.

[itamarst]

OK I think it's good now. Note that auto-merge won't happen because I edited something, you'll need re-enable it if you're happy.