Add support for nanobind?

[benbovy]

After a quick inspection of xtensor-python's internals, it looks like it shouldn't be too hard supporting both pybind11 and nanobind via some sort of minimal compatibility layer? Both have a pretty similar API for the basic features, and unless I'm missing something xtensor-python doesn't seem to depend much on pybind11's numpy and/or other advanced features?

Nanobind is more performant than pybind11 and also offers more control via its low-level interface, which would make things much easier for my use case (*). However, nanobind's numpy support is currently limited, e.g., it doesn't provide a vectorize helper (not sure it will anytime soon?) which is something that I also need. Xtensor-python would nicely fill this gap I guess.

(*) More context: I'm working on a Python/Numpy library (https://github.com/benbovy/s2shapely) providing bindings for the s2geometry / s2geography libraries, via vectorized functions (ufuncs) operating on Geography objects (C++ wrapped classes) referenced in numpy arrays with the numpy.object dtype.

[tdegeus]

I believe that currently this is the blocker : xtensor-stack/xtensor#2366 . It would be amazing if we could get that out of the way. Any help is greatly appreciated!

[tdegeus]

@benbovy I've been using C++17 fine. I would be very much open to support this. Have you had time to experiment?

[benbovy]

@tdegeus good to know C++17 works well. I haven't had the time yet to experiment with this, unfortunately.

[tdegeus]

Understood. Open to a contribution if you are up to it (I guess that ideas could be borrowed from https://github.com/wjakob/nanobind/blob/master/include/nanobind/eigen/dense.h ). Otherwise it will come when one of us finds the time or sufficient urgency

[tdegeus]

Just for the record: I did some experimenting with nanobind's ndarray which should be very similar to what we want, and things like allocation and function calls are indeed way faster (at least a factor two in what I experimented with), and compilation much much faster. I hope to get some time soon to work on this (and I hope even more that someone beats me to it ;))!

[Roy-Kid]

Some off-topic discussion: Is it possible to provide ctypes support? Since numpy has ctypeslib, I believe it should be very easy to provide a wrapper? And according to many benchmarks, ctypes performs better than pybind: taichi-dev/taichi#4830

[SebastianThiede]

Just for the record: I did some experimenting with nanobind's ndarray which should be very similar to what we want, and things like allocation and function calls are indeed way faster (at least a factor two in what I experimented with), and compilation much much faster. I hope to get some time soon to work on this (and I hope even more that someone beats me to it ;))!

I did a bit on this for my work. Didn't test for xarray, only xtensor and xtensor_fixed. Maybe a starting point for you?
SebastianThiede/nanobind_xtensor

Some off-topic discussion: Is it possible to provide ctypes support? Since numpy has ctypeslib, I believe it should be very easy to provide a wrapper? And according to many benchmarks, ctypes performs better than pybind: taichi-dev/taichi#4830

Btw. Nanobind can also be very fast compared to pybind11.

[Roy-Kid]

Just for the record: I did some experimenting with nanobind's ndarray which should be very similar to what we want, and things like allocation and function calls are indeed way faster (at least a factor two in what I experimented with), and compilation much much faster. I hope to get some time soon to work on this (and I hope even more that someone beats me to it ;))!

I did a bit on this for my work. Didn't test for xarray, only xtensor and xtensor_fixed. Maybe a starting point for you? SebastianThiede/nanobind_xtensor

Some off-topic discussion: Is it possible to provide ctypes support? Since numpy has ctypeslib, I believe it should be very easy to provide a wrapper? And according to many benchmarks, ctypes performs better than pybind: taichi-dev/taichi#4830

Btw. Nanobind can also be very fast compared to pybind11.

https://github.com/yanto77/cpp-python-binding-benchmark This is another benchmark I found recently